Just A Summary

Why am I asking?

Recently, Typo, the ruby based blogging engine that this site runs on, underwent a strutural change that involved replacing 4 different tables in the database with a single table, and migrating all articles, comments, trackbacks and pages into this new table.

The migration was easy to write. Typo’s built on top of Rails, and Rails has tools for doing database migrations. It’s hard to test a migration though, so the first ‘big’ migration that moves articles into the new contents table just raises an exception saying “BACK STUFF UP before uncomment this line in the code and rerun the migration.”

Making that backup is a really good idea because, if you’ve turned on spam protection, the next migration accidentally throws away all comments on sufficiently old articles.

Oops.

When a comment is saved, it first ensures that the article it’s attached to isn’t too old (more than 14 days old by default). If it is, the comment doesn’t get saved. This is fine when you first attach a comment to an article, but less fine when you’re, say, adding a guid field to the article or, as in the case of the migration, moving the comment from one database table to another.

So, I’m adding a test or two to the test suite to ensure that we can modify comments on old articles. Once I’ve got that passing, we can be reasonably confident that, next time we do a migration that targets comments, we can be confident that all the comments will get modified appropriately, which can’t be bad.

What have I learned?

Because the change was a refactoring which adds no new functionality to Typo, I hadn’t written any new tests, just made sure that all the existing ones were passing. But no test suite is complete (unless you are testing a “Hello, world” program I suppose), so the issue that causes us (and anyone else who migrated to edge typo with spam protection turned on) pain wasn’t caught by the tests.

I don’t blame whoever wrote the tests in the first place for missing this particular case; there’ve been occasions when I’ve wanted to edit an old comment to correct a URL or something and have bounced off the spam protection, but it never occurred to me that this might be a problem for migrations until I saw the bug report.

What I have been forcibly reminded of is that no test suite is complete. So, when a bug comes in, you turn it into a test and add it to the test suite, then you should never introduce the same bug again.

Nothing new there of course. Surely everyone already knows this rule of testing. But there’s nothing quite like actually doing it again, and having a good deal of confidence that adding the test will save you from an entire class of problems in the future to help you internalize it.

I think that, before I do much more work on Typo, I’ll be taking a closer look at what is and isn’t tested in its test suite.