Photo credit: stopnlook flickr

Not testing at all is at one side of a spectrum, at the other end is testing everything. Those two are easy picks to rule out since the truth is almost always somewhere in between.

I remember reading about one physicist building his own supercomputer. When asked why he did that he said something like this:

I could have just given 2M dollars to IBM. They have a great team and they would build a great supercomputer. But this way I get more bang for a buck.

This is a major issue. Getting the most bang from your testing buck.

In practice I’ve found that these are good enough approximations:

System core

Core is usually some kind of complex algorithm. It pays off handsomely to test this. Consider even putting it through mutation testing if you are in that kind of S&M.

Something scares you

This is a good sign that you have a fruitful place for tests.

You don’t have a clue

Great for testing. As a side effect you’ll get to find out the solution, rather then invent the solution. The best abstraction for given problem will crystallize before your unwilling eyes. You can steal a credit for it later on and brag about your superior brain power that led you to such elegant and simple solution.

Repeating problems

Something turns out tricky and bothersome in production. Great time to tighten up test suite around it. Blow the son of a gun out of the water forever. No more 1AM calls…

Front end

Just say NO! Unless you are a multimillion dollar corporation with hundreds of coding monkeys. For big system automation could pay off, for smaller projects cucumbering feels like Java.

Integration tests

You need some, but stay rational. I usually do the bare minimum.

Conclusion

My reasoning is this. You are not an idiot. You will rarely mess up obvious stuff. Testing that you have a link in a right corner is fragile, and well stupid. In theory it might be the “right” thing to do.

Whenever I did that it just felt wrong and it rarely saved my bacon.

What I want is:

• to be sure about hard parts
• ability to change my mind about implementing hard parts
• ability to add features to the hard parts
• ability to simplify the hard parts by throwing out stuff that I don’t use
• to test the minimum that will allow me to get away with it

As Amy Hoy would put it:

Do you want to be right or do you want to be effective.

History is full of failed projects that where “right”. So it will do you no good to be righteous about testing. It’s 25 anniversary of OS/2 so that comes to mind. Don’t forget other like Multics or micro kernel based kernel GNU Hurd.

I’ll let you in on a secret. They were not really better. They only seemed better on the first glance. Kinda like Titanic seemed unsinkable.

What is your opinion about this?