Valuable Information From TestResultsPosted: 24/09/2013
In the following example, I will show how to get meaningful information from tests. All examples are written in C# (.NET 4.0) and use NUnit 2.6.2 and TeamCity 8.0.
For all tests, the same system under test is used – to keep it simple, I just wrote one simple method. This method is used for all tests.
The basic idea is to create a CultureInfo from a string. As test input data, a list of strings is used.
The poor’s man approach – Execute all tests in a loop
This is the simplest way to do it: use a foreach-loop and fail if one or more tests are failing.
In this case, no information is available which test(s) were failing. A small improvement is to use NUnit’s overload methods of the Assert.Fail(), Assert.IsTrue() etc methods where a string can be provided:
One test per item – TestCaseSource
From NUnit 2.5, it is possible to use the TestCaseSource attribute. With this mechanism, a test is executed n times.
In our example, you see that there are 24 items, and 3 of them fail:
If a test cannot be executed n times within the same session, the test has to be executed once in n sessions (loop of tests). A typical reason might be if a test relies on a value of a configuration file.
Execute single tests in a loop
Since the tests are executed in a loop outside of NUnit, only one test is visible in NUnit GUI:
In TeamCity, the same MSBuild code is executed 24 times, each run with one test.
As a summary, have a look at TeamCity’s output for all 4 variations: