Build One to Throw Away

This is a maxim from Fred Brooks’ The Mythical Man-Month. These days I’m thinking it’s the single most important lesson there is about software. It’s been brought rudely home to me by my recent work on mod_atom, whose design is terribly simple; but I still got the first cut wrong in important ways.

What I Got Wrong · mod_atom is about as simple as an AtomPub server can be; all the entries and feeds and collections are backed one-for-one by ordinary flat files, with entries scattered across a YYYY/MM/DD tree, just like here at ongoing.

I had two URI/filesystem hierarchies, rooted at atom/ and pub/, to separate the resources that exist for AtomPub, like Service Documents and Media Link Entries, from those that are there to be read, like feeds and HTML pages; the goal was to make access control easy. How could that be wrong?

What happened was, I went back, after a few months of ignoring mod_atom, to build in Meta-pubs, so you can do CRUD on whole publications, not just individual entries and media files. This ended up touching most pieces of the system, and I realized that a couple of the core ideas were leading me into all sorts of tangled-code complexity; in particular the double directory structure.

So I rewrote it with one filesystem tree routed at pub/, backing two URI spaces which were identical except for being rooted at pub/ and atom/. The mod_atom code forces all PUT and POST and DELETE requests to go through the atom/ subtree. This allowed me to slash literally hundreds of lines of code.

There was another design error, just as obvious in retrospect; its cleanup also allowed subtracting severe gnarl and great big chunks of code.

Programmers experience soaring joy when they can rip through code deleting functions and declarations, screens-full into the bit bucket, with the steady drumbeat of tests-fail-then-pass.

So maybe I didn’t build one to throw away, but I built one that needed major amputations out of the box.

And Your Point Is? · This is by most standards a simple system; at the moment, excluding various XML- and HTML-munging libraries, less than six thousand lines of code. I’m an experienced programmer, have written hundreds of thousands of lines of C code, know the internals of httpd pretty well, and understand AtomPub as well as anyone in the world.

I’m not a truly great developer (I know this is true because I’ve worked with some) but I’m not a moron either (worked with some of them too) so if I’m going to make this kind of misstep, a whole lot of other people will too.

And let me make a strong statement: I’m not sure there’s anyone living, including your five favorite programming heroes, who could tell you, without writing the code, whether one or two directory trees were going to work out better for this problem.

Given that, what hope is there for waterfall development? Or for any approach that doesn’t leave space for going back and building things right once you’ve learned what “right” is by building things once? Well, none.

But You Already Knew That · Probably true. But you know what, how about we all agree, all of us who write about writing software, to write about this once every year or so. Because there’s this terrible glaring conflict between what sensible managers want and what sensible programmers know. Managers, good managers, want a plan; they want to lock in design constraints so that work can be dealt out and progress tracked and promises kept. Programmers, good programmers, know that they’re not smart enough to get the core design choices right until they’ve built something that works.

The various techniques and disciplines gathered around the banner of “agile” are on balance more honest at facing up to this unavoidable tension. But there’s still lots more work to be done.

And the most important thing is, we all have to remind ourselves, all the time, that we’re not smart enough to get anything important right the first time.

Exceptions? · Yes, of course. If you’ve just written three driver-scheduling systems or foreign-exchange systems in a row, you’ll probably go into the fourth with a pretty good grasp of what’s important. And those kinds of systems matter. But the interesting software is by definition the stuff that isn’t the fourth iteration of anything. Perhaps it’s best to close with Brooks’ quote in full:

“Where a new system concept or new technology is used, one has to build a system to throw away, for even the best planning is not so omniscient as to get it right the first time. Hence plan to throw one away; you will, anyhow.”

---

---

Contributions