NGUPLUS: Frustration grows when unit testing with VS2005

Tuesday, February 21, 2017

Frustration grows when unit testing with VS2005

Ive always liked unit testing as it makes my job as a software developer much easier. But the unit testing features of Visual Studio 2005 makes me very frustrated and tired of writing unit tests.

There are two big key issues when using unit testing in VS2005.

When I have executed a test and it fails; it will show up in a list with a red icon. When I double click on the failing row/test it will show a report stating what went wrong. Tell me something I dont know, and move the focus to the failed assertion.
When I have fixed a bug in the project under test, I would like to re-run the test. In the "Test results" window I can choose "Rerun original test". When I click on it, it will re-run the test with the old code; and the test will fail again even if I fixed the bug. What happened, didnt I fix the code??
What the h**l is "Assert.Fail failed."?

Visual Studio 2005:

Eclipse:

Issue 1
Since the test report window has already displayed that the test fail, Im not interested in a report stating that the test failed, I know already that the test failed and (hopefully) the reason for it. What I am interested in is which assertion failed and where it is.

In VS2005 I have to do the following to get to the assertion:

Double click the failure in the "Test results" window.
Find the row number by reading the text in the stack trace section. Add row number to memory.
Open the unit testing code file.
Go to the row found in step 2
Fix the assertion

In Eclipse it is so much easier:

Click on the failure in the "Junit" window.
Double click on the "AssertionFailed" in the "Failure trace"
Fix the assertion

Cmon; 4 steps and many user interactions involving remembering trivial values to show the failed assertion. It is really frustrating to do it whenever something fails. If something is frustrating and takes time, the developer will stop doing it; thus stop unit testing the software. There is no context menu item to go to the failed assertion, only to go to the begining of the test method. With test cases that can be long, it is not so easy to find the failed assertion. Below is the test report that is shown when the user double clicks on the failure.

Issue 2
I have one project that contains the production code, and one test project next to it as I dont want to mix production code with test code. Ive created a test project, added a reference to the production project, and verified that the test project depended on the main project. The test project can access the production classes, so far everything is fine.

But the problem comes when one of the tests fails because of a bug in a production class. When I update the code in the production project, and want to debug the last failed test I click on the "Debug original test". Sure the debugger starts, but the problem is that VS2005 forgot to build the whole solution including the production project. Instead of rebuilding the production project it will only rerun the test. The test will of course fail again, and I will be sitting there wondering if my fix didnt do anything. This has happen to many times, and Ive wasted numerous minutes before realising that I must manually build the solution.

Perhaps Im not using the features correctly, and MS has another view on old test results. Maybe test results should be saved every time a test is run. But for me test results has a very limited lifetime, as soon as Ive fixed an error; I dont care about the results prior to the fix.

This does not happen in Eclipse, the minute that Ive updated the code I can re-run the test by clicking on "Re-run Last Test - Failures first". No rebuild needed, no time spent wondering why a fix doesnt work.

Issue 3 (Minor)
I just saw this when writing this blog. How can an Assert.Fail() fail? The failure text states that the Assert.Fail() itself failed, to me that is not correct. The test failed itself, and should perhaps say "Failed due to an Assert.Fail() call". Unfortunately Eclipse is not much better as it states "AssertionFailedError" for a fail() call, but still that is better than VS2005.

Opinion
The VS2005 unit testing features are not user friendly, they dont help the user as good as they could. Sometimes a tool with a bad feature implementation is worse than a tool without the feature. Perhaps MS can improve the usability in VS2005, and make it easier for us developer that wants to utilise unit testing. It seems that the above design misstakes must be developed by people who doesnt practice unit testing.

Available link for download