Parameterized Tests

Note: The feature described on this page requires csUnit 2.2 or later.

Parameterized testing is sometimes also referred to as data-driven testing. csUnit supports parameterization of tests in several ways:

Simple Parameterization of a single test
Parameterization using a static method or property
Parameterization using an XML file
Parameterization using a database table

Which option you choose depends on the circumstances. The order in which they are listed here starts with the simplest case and ends with the most powerful scenario.

Parameterization can be used to test algorithms, APIs, and similar items.

All types for parameterization are currently located in the csUnit.Experimental namespace.

Simple Parameterization

Most tests can be written without the need of parameterizing them.

In some cases however, you would like to be able to test an algorithm that takes a number of inputs and produces some number of outputs. As a very simple example, let's use the calculation of a discount as a percentage of the invoice amount:

Invoice Amount	Discount Percentage
100	0
200	5
500	10
1000	15

This simple case is certainly not very thrilling, but you get the idea. Without parameterization you would have to write four tests, one for each invoice amount:

[Test]
public void DiscountRange1() {
DiscountCalculator calc = new DiscountCalculator();
Assert.Equals(0, calc.PercentageFor(100));
}

[Test]
public void DiscountRange2() {
DiscountCalculator calc = new DiscountCalculator();
Assert.Equals(5, calc.PercentageFor(200));
}

[Test]
public void DiscountRange3() {
DiscountCalculator calc = new DiscountCalculator();
Assert.Equals(10, calc.PercentageFor(500));
}

[Test]
public void DiscountRange4() {
DiscountCalculator calc = new DiscountCalculator();
Assert.Equals(15, calc.PercentageFor(1000));
}

Instead, it would be nice to refactor the obvious duplication within the tests and write a far simpler test. Parameterized tests allow you to do exactly that.

With parameterization, you add parameters to a test and tell csUnit where to read the parameter values from. To support this, we have introduced the attribute DataRow, which takes any number of parameters. You can use the DataRowAttribute to decorate parameterized tests. Each attribute corresponds to a single execution of the test.

In essence you have to give the test parameters and tell csUnit where to get the parameter values from. For that, we have introduced the attribute DataRow, which takes any number of parameters. You can use the DataRowAttribute to decorate parameterized tests.

Taking the example from above, we can write a single test with four DataRow attributes as follows:

[Test]
[DataRow(100, 0)]
[DataRow(200, 5)]
[DataRow(500, 10)]
[DataRow(1000, 15)]
public void DiscountRange(int invoiceAmount, int expectedDiscount) {
DiscountCalculator calc = new DiscountCalculator();
Assert.Equals(expectedDiscount, calc.PercentageFor(invoiceAmount));
}

Basically, we have refactored the four discrete tests into something easier to understand and maintain.

Specifying An ExpectedException

What if for some data rows you'd expect an exception to be thrown? Well, csUnit supports this as well, through a named property to the DataRow attribute, ExpectedException.

Here is a simple example of how an expected exception can be specified:

[Test]
[DataRow(2, 1, 2)]
[DataRow(6, 2, 3)]
[DataRow(4, 0, 0, ExpectedException = typeof(DivideByZeroException))]
public void DivisionTest(int numerator, int denominator, int quotient) {
Assert.Equals(quotient, numerator / denominator);
}

Note: more than one data row can have an expected exception. Also, the expected exception can be different for each of those data rows.

Parameterization With Static Method Or Property

The DataRow approach is useful if you only want to use a set of parameters once. However, in some cases you may want to use the same set of data for more than one test. In this case you can use the DataSourceAttribute and specify a type as a parameter for the attribute. That type is used as the data provider for the parameterization. It needs to implement a static method or a static property that returns an array of data rows.

[Test]
[DataSource(typeof(FixtureWithStaticDataProvider))]
public void Squares(int oper, int result) {
Assert.Equals(result, oper * oper);
}

This test requires a class with the name FixturewithStaticDataProvider to be implemented elsewhere. For example:

public class FixtureWithStaticDataProvider {
   public static DataRow[] GetTestData() {
      return new DataRow[] {
         new DataRow(0, 0),
         new DataRow(1, 1),
         new DataRow(2, 4, ExpectedException = typeof(DivideByZeroException)),
         new DataRow(3, 9)
      };
   }
}

At runtime, csUnit will search the data provider class for a static method that returns an array of DataRow objects. It will invoke the first one it can find and use the returned array of DataRow objects as the parameter sets for the parameterized test method. Please note that the data provider class and the test fixture containing the test can be the same.

Again, if you expect an exception to be thrown for one or more of the data rows, you can assign the expected exception type to the named property ExpectedException of the DataRow attribute. This can be seen on the third data row above.

Parameterization With XML-File

Suppose that you would like the parameters to be read from an XML file. You can do this with the DataSource attribute as well. Here is an example:

[Test]
[DataSource("Squares.params.xml")]
public void Squares(int oper, int result) {
Assert.Equals(result, oper * oper);
} The content of the XML file would then looks as follows:

   <dataTable>
      <dataRow>
         <value>1</value>
         <value>1</value>
      </dataRow>
      <dataRow>
         <value>2</value>
         <value>4</value>
      </dataRow>
      <dataRow>
         <value>3</value>
         <value>9</value>
      </dataRow>
   </dataTable>

And again, we also need to account for the possibility that one or more of the data rows expects an exception. Here is an example of such an XML file:

   <dataTable>
      <dataRow>
         <value>1</value>
         <value>0</value>
         <value>0</value>
         <expectedException>System.DivideByZeroException</expectedException>
      </dataRow>
      <dataRow>
         <value>4</value>
         <value>2</value>
         <value>2</value>
      </dataRow>
   </datatable>

Note: as with the other ways of specifying an expected exception, you can specify any type. This includes exceptions that you have implemented yourself.

Parameterization using a Database Table

The fourth option to provide sets of parameter values to a parameterized test is specifying a .NET data provider, a connection string, and a database table name. Here is an example:

[Test]
[DataSource("System.Data.SqlClient",
"Data Source=.\\SQLEXPRESS;AttachDbFilename=" +
   "\"C:\\data\\csUnitTestData.mdf\";" +
   "Integrated Security=True;Connect Timeout=30;User Instance=True",
   "DivisionTests")]
public void Division(int nominator, int denominator, int expectedResult) {
   int actualResult = nominator / denominator;
   Assert.Equals(expectedResult, actualResult);
}

As you can see this is just a standard connection string which csUnit passes on to the managed ADO.NET data provider. The first parameter is the Invariant Name of the .NET data provider. The factory for the data provider must be registered in the machine.config file. By default .NET has factories registered for SQL, Oracle, OLE DB, and ODBC.

The table name - "DivisionTests" in this example - refers to a table that contains the actual test parameters. The table content looks as follows (screenshot from VS2005's server explorer):

server explorer toolwindow screen shot

The table contains 3 rows each representing one set of parameters for a parameterized test.

The image also shows the table definition with all three columns having a data type of int. You can choose whatever datatype you want it to be, and they can all be different.

During runtime csUnit executes the test once for each data row and reports separately on the outcome.

Please note, that accessing a database, if local, is an expensive operation. Some databases work in-memory thus at least avoiding the cross-process and/or cross-machine communication. Whether you choose a database table as a feed for your parameterized test requires careful considerations and trade-offs.

Note: The feature described on this page requires csUnit 2.2 or later.

Sponsors:

Sources hosted by