Testing REST

I’ve been using RSpec in a way that’s probably inappropriate, but has got me thinking about Test-Driven Development and REST.

RSpec? · Rubyland is all about TDD, but if you listen to the Kool Kids, they don’t talk about “tests” or even “unit tests” any more, but about “specs”. Which comes from RSpec, a Ruby framework that is positioned as not for “unit testing” but for “Behavior-Driven Development” or BDD. I was first exposed to it in an April-2006 presentation by Dave Astels at Canada On Rails.

I’d explain BDD but that’s covered very nicely on the RSpec page and also in Bruce Tate’s Behavior-driven testing with RSpec. I’ve never actually managed to write any software with a pure BDD approach, but the closer I’ve come the happier I’ve been.

Now you could say that as frameworks go, RSpec is just a retake on xUnit with heavy lashings of syntax sugar, and I think there’s justice in that; but on the other hand, readability is important and I generally like syntax sugar.

The Trouble With mod_atom · I’ve been grinding away part-time on mod-atom for over a year, but I really have to get it somewhere near 1.0 so I can do performance testing so I’ll have something interesting to say at Apachecon even if I’m speaking late Friday thus to an audience of 8½ people.

The development of mod-atom has been test-driven, because The Ape existed before I started. But not, I’m afraid, unit-test-driven. In the early days I had unit tests for some of the low-level path-munging and directory-walking routines, but they were brutally difficult to maintain (I had to mock out essentially all of APR) so eventually I junked ’em. But I always had the Ape and Joe Gregorio’s AtomPub client keeping me honest.

However, recently I started going beyond the limits of what the Ape currently tests: publication CRUD, HTML generation, and app:draft support. The good news is that the Ape code has generic support for GETting and PUTting and POSTing and checking media-types and extracting links; it wasn’t actually designed to serve as a REST testbed, but with a bit of refactoring it’s given me what I need.

Some REST Tests · An example is worth a thousand words:

  it "Should create a sub-pub if posting to a meta-pub" do
    m = meta(@meta)
    c = Ape::Collection.new(m.href)
    entry = new_entry(:title=>'Posting to meta',
      :summary=>'To see if it works', :content=>'Does this matter?')
    p = c.post( :data => entry, :slug => 'sub-pub-title')
    p.should be_kind_of(Ape::Entry)

    # find the <link rel="publication"
    @sub = p.link('publication')
    exists?(@sub, 'application/atomsvc+xml').should be(true)

@meta is the URI of the meta-pub Service Doc. There are a few utility routines, obviously. Some are built into the Ape like Entry#link, others were cooked up for this purpose but probably belong in a generic framework.

Here’s another:

  it "Should respect app:draft" do
    ce, c = entries(@pub)

    entry = new_entry(:title=>'Just a post',  :draft => 'yes',
      :summary=>'Just text', :content=>'Wanna see text')
    p = c.post( :data => entry, :slug => 'Want Text')
    p.should be_kind_of(Ape::Entry)
    
    h_uri = p.uri.sub(/\.atom$/, '.text')
    @to_clean_up << p.uri
    exists?(h_uri).should_not be(true)

    # now make sure it also isn't in the feed.atom or index.html
    feed_uri = ce.href.sub(/collection.atom$/, 'feed.atom')
    h = Ape::Getter.retrieve(feed_uri, 'application/atom+xml')
    h.should be_kind_of(Net::HTTPSuccess)
    rel_uri = h_uri.sub(%r{^http://[^/]*/}, '/').sub('/atom/', '/pub/')
    h.body.index(rel_uri).should be(nil)

In this one, there’s some cheating happening; the test code knows what what the resource names created by mod-atom look like and peeks into URI space to see if they got created. And it looks to see if there are links to the shouldn’t-exist public version of the resource by just scanning representations for the URIs.

Anyhow, URI-driven: check. Resources and representations: Check. Hypertext as engine of application state: check.

What’s Wrong With This Picture? · Well, it might be RSpec, but it sure ain’t BDD. As Headius pointed out when I was telling him about this, it’s actually functional not unit testing. And in fact I wrote a ton of C code first and the tests later. And furthermore, either I’m not using RSpec right (quite likely, it’s my first time) or it has an impedance mismatch with the notion of working with a stateful system, where you create resources in one test and interact with them in the next and destroy them in a subsequent one.

But hey, it sure helped me with mod-atom and it’s pretty readable and it wasn’t too much work to sketch in the tests. So I’m managing to restrain my guilt.

What Would RESTUnit Look Like? · I’m early in this thought process, but some basics occur to me:

• Access to GET/POST/PUT/DELETE with fine control over the headers, and utilities to wrangle ETags and so on.

• Easy XPath for pulling stuff out of messages which happen to be in XML.

• Convenience functions for running around the DOM digging URIs out of message bodies.

• Possibly some Atom-specific stuff, on the assumption that a lot of the payloads in RESTful conversations will be Atom. For example, you really need Entry#link(rel).

• A place to stash information about the resources you’ve created and are interacting with, and a cleanup function so you leave the URI space the way it was when you started.

As an RSpec newbie, I get the feeling that a lot of what I need would be easy to add on or may already be there.

What Other People Are Doing · I thought “Surely others have thought about Test-Driven RESTful Development” and went poking around, but didn’t find much. There’s RESTClient, but it seems more like a debugging aid.

Then of course there’s Joe Gregorio’s APP Test Client; Joe tells me that, just as I’ve been doing with the Ape, he’s been pulling apart the generic-AtomPub-client and AtomPub-test-driver pieces, so I suspect that it’d be at least as good a basis for TDRD as the Ape.

Aside from some Rails-specific stuff here and there, that’s all I’ve turned up. Let me know I’m wrong in the comments below.

Which is odd; the combination of TDD or BDD and REST feels natural to me; write code to exercise the service, then start filling in the service.

What Am I Going to Do? · Not over-think it, for now. I still have a lot more tests to write and then I need to build a bunch of scripts to exercise performance... hmm, seems that a RESTful test suite and performance exerciser might share a lot of code. When I get that done, I’ll think about it and develop an opinion as to whether there’s maybe a BDD framework in there trying to get out, or at least signposts for what one ought to look like.

Or not; It’s perfectly possible that there’s no sweet spot here, that the essential details of REST apps differ so much that there’s not really much shared testing framework to pull out.

But I think the idea is stimulating enough to share while still half-baked.

---

---

Contributions