I guess I need to supply some context here: I’m still in the JavaScript world and I’m still writing desktop-like applications that run in the browser. I only load the DOM once. Here we go:
Every test you add to your application creates a dependency on it. More tests mean more dependencies. Dependencies spell trouble, especially if you depend on code that changes a lot, a.k.a. unstable components.
If you have a test to verify the correctness of a feature, the test should touch a minimal amount code, and it should ideally only touch stable components.
A common problem with integration tests is that they break for reasons having nothing to do with the feature they are testing. You often run in to this problem if you are testing thru the UI. The UI is very unstable (it changes a lot). Minor design changes can break tests. If you cannot easily change your application without creating extra work, the ROI of the integration tests is diminishing.
One solution is to ignore the outer layer of the UI and instead test one layer in. When I write desktop-like web applications I have a thin view layer that encapsulates all the UI interaction (http://bjarte.com/post/effective-javascript-unit-testing). This means that I can invoke every user action thru the view layer.
Let’s look at the following example of a search box:
The view would look something like this:
function SearchView(callback) {
$("#search").click(function () {
callback($("#q").val());
});
function doSearch(){
$("#search").click();
}
return {
doSearch: doSearch
}
}
Somewhere else in my code I will make all my views visible globally thru a “views” collection. From my tests I can now invoke behaviour by talking directly to the SearchView:
views.searchView.doSearch();
I actually use WatiN to test thru the browser. The only line of code I need in my test (actually specification) to invoke the behaviour is
Browser.Eval("views.searchView.doSearch();");
As you can see, some of my application code is there only to support integration tests (e.g. the doSearch method on the view). I don’t have a problem with that.You can plug a computer into a modern car to diagnose it. Why shouldn’t you be able to diagnose software?
The point here is that my tests depend on more stable code. The layout can change. The button can move, or perhaps become an image. The search behaviour will stay the same. Hopefully.
Obviously you are not testing EVERYTHING when you don’t run the tests thru every layer of the application. But you know what ? You cannot test everything anyways. You come to a point where writing more tests will not give you more benefits. At some point more tests become a problem, a maintenance problem. Where you draw this line is dependent, not only on your technical challenges, but also the people in your team, the culture, the process and many other factors. Context, context, context.
Later.
