Just A Summary

One aspect of coming back to Perl for ‘recreational’ programming is that if, like me, you’ve declared war on @_ and boilerplate code, then testing can be somewhat trying. The Perl testing framework that best fits my head is Test::Class, which is an excellent Perlish implementation of xUnit style testing. If you’re unfamiliar with the, library, Ovid is writing what’s shaping up to be an excellent series of introductory articles about it at http://www.modernperlbooks.com/.

The problem I’m having with Test::Class at the moment is that I can’t write:

use MooseX::Declare
class Test::Person
  extends Test::Class
{
  use Test::Most;

  method class_under_test {'Person'}
  
  method startup : Test(startup => 1) {
    use_ok $test->class_under_test
  }

  ...
}

Test::Class is doing too much in its initialization phase, and relies too heavily on code attributes, for it to play well with MooseX::Declare. Drat.

On reflection though, this might be a good thing, because maybe MooseX::Declare isn’t really what’s needed. What I’d like to write is something like:

use ...;

testclass Test::Person
  exercises Person
{
  startup class under test should be usable (1 test) {
    use_ok $test->class_under_test
  }
}

And have the library ‘…’ expand the testclass declaration into something that looks like the first code snippet. After all, if MooseX::Declare can work without source filters, it should be possible to come up with something nicely declarative for specifying test classes.

Obviously, there’s nothing on CPAN that does this yet though. So I went fossicking through MooseX::Declare to see how it works¹ and discovered thing of Lovecraftian beauty that is…

Devel::Declare

Devel::Declare is possibly the most hostilely documented library I’ve ever come across. Its documentation only begins to make sense when you already understand enough about how it works that you don’t really need the docs. What it does is to let you declare your own Perl keywords. You could, for instance use it to introduce given/when into versions of Perl that don’t have it yet. You declare your keywords and associate them with parsers. When, during its compilation phase, perl hits one of your keywords in the right context, it hands off to your parser which can then do what the hell it likes in the way of code transformation, before handing control back to Perl, which then parses the transformed code as if that was what was there all along.

So, to want to transform that testclass syntax I just pulled out of my ass into a real Test::Class package, I just need to write an appropriate parser and code generator, perform the appropriate Devel::Declare incantations, and I’m laughing.

Making progress

So far, I’ve got to the point where I have a working testclass keyword, but nothing yet for the ‘inner’ bits (setup, test, teardown, etc). I can write:

testclass Test::Person
  exercises Person
{
  ...
}

testclass Test::Person::Employee
  extends Test::Person
  exercises Person
{
  ...
}

and, as I write this, I’m realising that the syntax I’d cooked up for using extra test helper modules:

testclass AnotherTest
  helpers -More, -Exception, Carp
{
  # use Test::More;
  # use Test::Exception;
  # use Carp;

  ...
}

would probably read better as:

testclass AnotherTest
  +uses Carp
{
  # use Test::Most;
  # use Carp;

  ...
}

and also that I want this:

testclass exercises Person {
  ...
}

to build me a Test::Person class.

What’s still blowing my mind about Devel::Declare’s possibilities is that I’m no longer constrained to writing a Domain Specific Pidgin which works by building a tower of proxy objects and weird evaluation contexts to produce something that’s legal code in the host language, but which has the feel of another language. With Devel::Declare, I control the horizontal and the vertical until I choose to hand control back to Perl. Right now that means my error reporting is disgracefully bad, but it also means that I can roll a syntax that makes sense without worrying about how I’m going to get perl to parse it.

One of the things I find frustrating about writing RSpec specifications is that describe and it both want to be first class keywords – it feels like you should be able to write:

describe SomeClass, "in some context"
  before each
    # set things up
  end
 
  it "should do something or another"
    ...
  end
end

But, because RSpec works by taking advantage of Ruby’s block magic, you have to write:

describe SomeClass, "in some context" do
  before :each do 
    # set things up
  end

  it "should do something or another" do 
    ...
  end
end

I definitely prefer the version without the extraneous dos and the gratuitious : before each in the before declaration. Does anyone feel like writing devel/declare.rb?

Show us the code!

If you want to see the current state of my Test::Class::Sugar art, the place to look is http://www.github.com/pdcawley/test-class-sugar. At the time of writing it relies on http://www.github.com/rafl/devel-declare and doesn’t have anything so useful as documentation, a Makefile.PL or even any tests beyond the collection of code samples that is t/initial.t. Expect all those when and if I push it to CPAN.

Caveats

Yes, I know that this sort of metasyntactic abstraction is trivial in a Lisp. I just happen to like syntax, okay?

Update 20090312

use Test::Class::Sugar

testclass exercises Foo 
    +uses -Warn
{
    ...
}

Now generates

{
  package Test::Foo;
  use base qw/Test::Class/;
  use Test::Most;
  use Test::Warn

  ...
}

So that’s one hurdle jumped. And I now know how to write the various method helpers and, when I get the appropriately shaped tuits, I shall actually write the damned things.

Then all I have to do is document it.

And write up a proposal about it for YAPC::Europe.

Update 20090314

I now know what a plan looks like:

test with multiple assertions << 3 {...}

And, more importantly, I’ve implemented, and documented everything and am almost good to cut a 0.001 distribution. I need a few ducks up on CPAN, but once that’s done, we’re good and I can get on with parameterizing some of the assumptions that are hard coded at the moment.

¹ Something I swore blind I wasn’t going to do in my London.pm presentation. Seems my word isn’t to be trusted…

If you follow the Ruby blogs, you will probably have seen a bunch of programmers attempting to do something akin to Haskell’s maybe, or the ObjectiveC style, message eating null.

Generally, by about the 3rd time you’ve written

if foo.nil? ? nil : foo.bar 
  ...
end

you’re getting pretty tired of it. Especially when foo is a variable you’ve had to introduce solely to hold a value while you check that it’s not nil. The pain really kicks in when you really want to call foo.bar.baz. You can end up writing monstrosities like (tmp = foo.nil? ? nil : foo.bar).nil? ? nil : tmp.baz (actually, if you were to write that in production code, you probably have bigger problems). One option is to just define NilClass#method_missing to behave like its Objective C equivalent, but I’ve never quite had the nerve to find out how that might work. I wanted to write

if maybe { foo.bar.baz }
  ...
end

and have nil behave like an Objective C nil for the duration of the block, but no longer. So I wrote it. Then I thought about how to present it. I wrote the thing test first using rspec and the whole thing just flowed, but writing up a test first development process for a blog entry is painful, so I’ve made a very rough (but blessedly short) screencast of the process instead.

That’s a slightly reduced thumbnail, the movie is substantially more readable. The bottom pane of the window is the output of autotest rerunning the spec every time either the spec or the implementation changes. The top pane alternates between the specs and the implementation. Generally, every time I edit the specs, a test starts failing and every time I edit the implementation it starts passing again. In the (any) real coding run, there were of course false starts, but generally the specs kept me pretty straight.

A word or two of warning: This is a completely unedited, silent, screen cast, there are typos, backtrackings and other embarrassments. I stopped recording once I’d got 4 tests passing, but this is far from release quality (it’s perfectly usable if you know its limitations, but it’s not entirely robust).

Please let me know what you think of this. I’m aiming to make a more polished version, complete with voice over and it would be good to know which bits are confusing and need addressing in more detail in the voice over.

I’ve been eyeing the rspec and rspec on rails packages and thinking I should give them a go.

To my eye at least, something like:

context 'Given a published article' do
  fixtures :contents

  setup { @article = contents(:published_article) }

  specify 'changing content invalidates the cache' do
    @article.body = 'new body'
    @article.invalidates_cache?.should_be true
  end
end

context 'Given an unpublished article' do
  fixtures :contents

  setup { @article = contents(:unpublished_article) }

  specify 'changing content keeps the cache' do
    @article.body = 'new body'
    @article.invalidates_cache?.should_be false
  end
end

reads far more fluently than the equivalent Test::Unit based tests:

class CacheSupportTest < Test::Unit::TestCase
  fixtures :contents

  def test_changing_published_article_invalidates_the_cache
    art = contents(:published_article)
    art.body = 'new body'
    assert art.invalidates_cache?
  end

  def test_changing_unpublished_article_keeps_the_cache
    art = contents(:unpublished_article)
    art.body = 'new body'
    assert ! art.invalidates_cache?
  end
end

So, I installed everything and started to work on a new class in Typo using rspec. Rather annoyingly, this seemed to break the current test suite, so instead of working on my new model class, I set to porting the existing suite.

And, on about my third test suite, I found what I think is a bug in the suite. I’m not sure it’s a bug, because, the way the test is written (by me, I admit it), masks the intent quite dramatically. I’m also finding that the freedom to name specifications and contexts in English rather than method_names_that_go_on_for_ever is forcing me to come up with much more useful descriptions of what I’m testing. I find myself working on making the spec runner output read reasonably well as English, and doing that casts light on what is and isn’t being tested.

I’ve known for a while that Typo’s test suite is, um, spotty, but the porting process is really helping me get familiar with what’s being tested. I’m half tempted to start adding extra specs as I go, and if I could work out how to keep the existing tests working while I did it, I would, but my priority for now is to get to the point where I can check the specs and be confident that the new specs are no worse than the old tests.

Because I’m much more confident that I know what the specs are doing, I’m also confident that it won’t be hard to revisit them to help specify typo’s behaviour better. I’ll just have to give myself the discipline of beginning each coding session with half an hour of fleshing out the specifications before I get back to adding behaviour.