Punishing Test Coverage

Looking around for something complicated to test that folks actually use, I decided to pick on Lucene.NET. My idea was to make sure the code coverage works well on what is a complex piece of software with some really complex tests – and that it isn’t so painfully slow that nobody would ever actually use it.

I ran the tests in Lucene 1.9rc1. After a little bit of setup, and changing the NUnit references to 2.4 using the Fix NUnit References Option in the Test Explorer I ran the entire solution’s Lucene Test solution’s test using the TestExplorer. 301 tests with coverage ran in 3 minutes 9 seconds. There were a few failures in there – looks like the Lucene test run at least a few web connections that I didn’t set up my local IIS to respond:

As a comparison, I ran again with coverage disabled, I just hit the toggle button in the Test Explorer. 2 minutes 8 seconds.

Coverage does have some overhead since it computes activity on each individual line, but not an overwhelming overhead. I’ve had profiler/coverage experiences in the past that have 400% to 800% overhead – this one is just 50%.

  Permalink |  Comments[0]


Recent Entries

 » Build 2687 Posted
 » New Unit Testing Book
 » Build 2293 Update Posted
 » Microsoft Team System NUnit Adapter
 » MS Team System Test Support


You can leave a response, or trackback from your own site.

Leave a Reply

 
 

You must read and type the 5 chars within 0..9 and A..F, and submit the form.

  

Oh no, I cannot read this. Please, generate a