Testing Your Assumptions

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 04 Dec 2005 09:37:00 -0600

One of the joys of writing applications using Ruby on Rails is the way the framework is constantly evolving better ways of doing stuff. It’s one of the dangers too.

Each release of Rails brings new and groovy features to the table, so it’s nice to stay close to the edge. However, when you do that, a change in the framework can bring your whole app crashing down because a key assumption you made turns out not to be true any more.

This is especially common when the assumed behaviour is undocumented, or a surprising consequence of behaviour that is documented.

When this happens, your test suite can have failures everywhere; it’s hard to work out precisely what the underlying problem is.

So, what to do?

Easy. Write tests that express your assumptions. It’s always good to make assumptions explicit. It’s better still to make them executable. Whenever you use something that’s a little bit surprising or hard to find in the docs, or when you do something that works around a bug, write a test.

Then, when the framework changes and tests are failing all over the place, test your assumptions. Hopefully there’ll be one or two failing tests there that point you to a solution to your problem. If there aren’t, then at least you know what’s not causing the problem. And, when you do find the underlying problem, you can add the new assumption to your tests for later.

I’ve just been working on Typo and found (rather usefully as it happens) that, if you add counter caching to a model, the model that holds the cache field will call all its save/update hooks whenever the cache field is updated.

Which is a little odd; it could be argued that simply adding what should be an ‘invisible’ cache field to a model shouldn’t go changing its behaviour in this way. So, I’ve created an assumptions_test.rb file and written the behaviour up as a test. Now, if David Heinemeier Hansson decides that this is undesirable behaviour and changes it, we’ll be able to see what’s going on.

The idea (as is so often the case) comes from Kent Beck. In this case, from Test Driven Development. One more book by Beck that deserves to be on every programmer’s bookshelf.

Just A Summary : Tag testing, everything about testing

Testing Your Assumptions