While developing a new release of the CodePlex.Diagnostics exception and logging framework I encountered a scenario where it became very difficult to write an effective unit test. In the latest release of the framework developers will interact with the framework using C# extension methods on the System.Exception, System.String, and System.Diagnostics.Process classes. Writing unit tests for the publication of exceptions and the writing of log entries, the extension methods on the System.Exception and System.String classes, presented no problems although the extension method for the System.Diagnostics.Process class expects the process to have exited and this was the scenario that was causing the difficulty.
Initially I thought that I had three options to enable the extension method on the System.Diagnostics.Process class to be unit tested and they were:
- Using .net reflection alter the internal state of the System.Diagnostics.Process class such that the HasExited, StartTime, ExitTime, and ExitCodeproperties all return appropriate values, giving the illusion that the vstesthost.exe process has exited while in reality the process is still executing unit tests.
After exploring the internals of the System.Diagnostics.Processclass with Lutz Roeder's .NET Reflector, and writing some exploratory code altering the internal state of the class using reflection, it became apparent that this approach would not work. Within the System.Diagnostics.Process class there is an internal method called EnsureState, and while I was able to successfully alter the state of the instance such that the HasExited, ExitTime, and ExitCodeproperties all return appropriate values, an exception is thrown by the EnsureState method when the StartTime property is called.
With Visual Studio 2008 support for source debugging I was able to step into the EnsureState method, if you're not familiar with this feature then see my previous blog post on Visual Studio 2008 SP1 .NET Framework Source Debugging.
So after determining that option 1 wasn't really an option I then had to consider option 2 although unit testing frameworks and the tools that execute the tests created with those frameworks are generally not intended for dealing with processes and other system level programming concepts. Creating another process within a unit test and having the vstesthost.exe process observe the Exited event of that process just seemed like it was getting into a corner case with the Visual Studio unit testing framework and I determined that option 2 was not really an option either.
An object mocking framework just seemed to be the appropriate direction to unit test the C# extension method for the System.Diagnostics.Process class and unit testing was not for an open source solution it may actually have been option 1 as it really is the most obvious choice. Given that the CodePlex.Diagnostics framework is open source I really wanted to avoid introducing any dependencies that are not core .NET types.
Initially I decided to explore some of the open source mocking frameworks such as NMock and Rhino Mocks although the consensus on the blogosphere (See posts by fellow Microsoft MVP's Mark Michaelis and Roy Osherove) was that TypeMock Isolator presented additional capabilities that made it the compelling choice. Microsoft Research also has two related projects that might be worth following, Stubs and Moles.
TypeMock Isolator enables us to mock an object using the IFaker interface which is exposed through the Fake property on the Isolate class. Calls to the HasExited, StartTime, ExitTime, and ExitCode properties are then isolated from the System.Diagnostics.Process class by passing lambda expressions to the WhenCalled method and stating an alternate value to be returned using WillReturn. See the code within Visual Studio 2008 and the execution of the unit tests within the next two screen shots. SQL Server 2008 Management Studio can then be seen in the third screen shot and the process exit details can be seen in the last set of results.
It is not often that technology amazes me although TypeMock Isolator truly amazed me in what new scenarios it enables in unit testing and I believe it makes the testable easier to test and the untestable testable.
If you download the latest change set for the CodePlex.Diagnosticsframework from the Sources tab on the CodePlex website the you'll have the code shown below and I would highly recommend taking a look at TypeMock Isolator for your next project. CodePlex.Diagnostics 4.0 will be entering beta next week and there will be a second beta release in January followed by the RTM build sometime in Q1 2010.
I'll be blogging more about the CodePlex.Diagnostics framework and TypeMock Isolator over the next few weeks so stay tuned...