Unapocalyptic Software

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops · When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say? · First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then? · Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better · Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing · The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing! · Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day · It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

---

---

Contributions