Valuable Information From TestResults

In the following example, I will show how to get meaningful information from tests. All examples are written in C# (.NET 4.0) and use NUnit 2.6.2 and TeamCity 8.0.

For all tests, the same system under test is used – to keep it simple, I just wrote one simple method. This method is used for all tests.

The basic idea is to create a CultureInfo from a string. As test input data, a list of strings is used.

The poor’s man approach – Execute all tests in a loop

This is the simplest way to do it: use a foreach-loop and fail if one or more tests are failing.

Image

Image

 

In this case, no information is available which test(s) were failing. A small improvement is to use NUnit’s overload methods of the Assert.Fail(), Assert.IsTrue() etc methods where a string can be provided:

Image

Image

One test per item – TestCaseSource

From NUnit 2.5, it is possible to use the TestCaseSource attribute. With this mechanism, a test is executed n times. 

In our example, you see that there are 24 items, and 3 of them fail:

Image

Image

 

If a test cannot be executed n times within the same session, the test has to be executed once in n sessions (loop of tests). A typical reason might be if a test relies on a value of a configuration file.

Execute single tests in a loop

Since the tests are executed in a loop outside of NUnit, only one test is visible in NUnit GUI:

Image

In TeamCity, the same MSBuild code is executed 24 times, each run with one test.

Image

As a summary, have a look at TeamCity’s output for all 4 variations:

Image

 

 

Advertisements