LinqAF: Testing

This is part of a series on LinqAF, you should start with the first post.

How is LinqAF tested?

Cats testing claws.

In comparison to LINQ-to-Objects, LinqAF has a much larger surface to test – every enumerable implementation (of which there is at least one, and often more than one, per LINQ operation) exposes more than 3,000 methods.

Every operation has its own test class (using Microsoft’s UnitTestFramework), and includes at least the following tests:

• Universal – which tests that every enumerable exposes the expected methods for the operation.  I.E. AnyTests.Universal confirms all enumerables have Any() and Any(Func<T, bool>) methods.
• Malformed – which tests that every operation properly fails if called on a malformed (typically obtained via default(T)) enumerable.
• Chaining – which tests that every operation performs as expected when called on each enumerable.

Most operation’s test class also has an Errors test, which assures that every enumerable properly handles the error cases (like null arguments) in a consistent and correct manner.

Each operation also has additional tests that cover simple invocations (typically on a single enumerable), for easy of debugging.

Finally, I pulled in the entire Edulinq test suite.  This covers all the little edge cases of a proper LINQ implementations, kudos to Jon Skeet yet again.

What’s the LinqAF.TestRunner project for?

Visual Studio’s test runner process falls over if you attempt to run the entire LinqAF test suite – a number of the tests create lots and lots of types, which eventually results in an OutOfMemoryException in the 32-bit process.

TestRunner runs every test in a separate process, in addition to gathering coverage statistics using OpenCover.  I use Visual Studio while bug fixing, and the TestRunner project when confirm that a build is ready for release.  I don’t use TestRunner during normal development because it takes between several hours and a day or so to run (depending on the machine I’m using).

What’s next?

In the next post, I’ll cover how I’ve benchmarked LinqAF as well the results of said benchmarking.