What Programmers Do

I contributed a morsel of code, connective tissue linking two moderately-popular pieces of publicly-available software. The technology and culture that enable this? They’re the water and computer programmers are the fish, we can’t see it any more. By an accident of history I could this time; and want to write about it.

Sidebar: For non-geeks · Whenever you’re sitting in front of a computer or fiddling with your mobile, you’re interacting with software. How that software gets built, and the culture out of which it rises, are processes that affect your life; just like the tech and culture and people that manage airline schedules and commute-route closures and TV programming. People seem to like looks into the engine rooms of the cruise ship and the control tower at the airport, so consider this one of those.

Are you in editorial somewhere mainstream-ish? I’d love to do a version of this for you.

Sidebar: For geeks · Tl;dr: I wanted to add Keybase.io support to an Android app, struck out with my first choice because GNU AutoHell, found OpenKeychain, no C++ and clean as clean, easy to understand, nice friendly community, also the Keybase API is a straightforward HTTP/JSON thing. So it wasn’t rocket science, they’ve taken three pull requests so far, end of story, no biggie. Hadn’t done much GitHubbing because Sun and Google, so the experience was fresher for me than for most of you.

The explanations of what we do that I’ve read in the mainstream are pathetically lame; I’m trying for better.

What software is · An Android app lives in an ordinary file whose name ends in “.apk”, so we call them APKs just like everyone talks about PDFs or GIFs. After you’ve made your APK you plug your Android into your computer with the same USB wire you use to charge it, then type a special command into your computer to install the APK on it, then you run it. After you’re sure it works, there’s a publisher’s page at the Play store where you can upload your APK for the world’s pleasure.

The software I worked on · In this particular case it was an Android app called OpenKeychain, you can get it in the Play store.

When Ed Snowden initially reached out to journalists, he opened with “What’s your encryption key?” and they answered, more or less, “Huh?” Because high-quality encryption is free and it works. But at the moment it’s overly complex to set up and use.

There’s pretty good free software out there called GPG that will do encryption and decryption and so on for you, and OpenKeychain makes it available on Android devices in a reasonably straightforward way.

There’s some new software out there called Keybase, which provides an easy-to-use, trustworthy directory of people’s encryption keys; I wrote about it in March. So if the journos Snowden reached out had had OpenKeychain and Keybase, life would have been easier.

What I did was hook up the OpenKeychain Android app to the Keybase directory. Because I think it’s important that people be able to communicate privately, and this might make that a little easier.

At the right is a screenshot showing off my contribution. A reasonable person would wonder what all those numbers in the list mean, and I would agree; the crypto people need to learn that if they want to appeal better to the mainstream, they need to hide all that gibberish. Lots more work to be done around here.

What it’s made of · That APK is assembled from lots of different pieces. Some of them are the actual code, instructions for the computer, in this case written in the popular Java programming language. Then there are a bunch of “XML” files that are used for housekeeping tasks like describing screen layouts and multilingual strings and and so on. Finally, there are little bits of multimedia; the graphics and videos and sounds that appear in the app.

We call all these pieces the “Source Code” for the app.

There are a lot of pieces: At this moment, OpenKeychain uses about 3,600 Java files, about 1,700 XML files, and about 3,100 media files. Impressed? You should be; those are big numbers. But it turns out that most of them aren’t exclusively OpenKeychain stuff, but pieces of software “libraries” written by others and used by many projects, not just OpenKeychain; to help display graphics, do the hard crypto math, and interact with the person holding the phone. The actual OpenKeychain-specific bits are 150 Java files, 853 XML files, and 2,064 graphics files.

That’s still quite a few. The huge number of graphics files is typical of Android apps; it’s big because Android devices come in so many shapes and sizes and screen qualities; something as simple as an on-screen button graphic might need to exist in a dozen different sizes and densities to look good on all of them.

Assembling all the pieces to make the APK takes a half-minute, more or less, on my pretty-fast MacBook Pro; a good occasion to visit the bathroom, check Twitter, or whatever.

How it’s made · Programmers use software to build software; we are our own customers. We have lots and lots of tools to organize and edit and analyze and combine and assemble and package all those pieces into that APK file.

Most people will never deal with these, any more than I’ll ever operate an air-traffic-control terminal or a steel-mill console or a derivatives-trading screen. But now at least you can say you’ve seen one; here it is.

That particular screen is from a tool called Android Studio; programs that help programmers program are among the biggest and most complicated pieces of software in the world, and represent thousands of person-years of engineering effort by some of the most talented members of our profession.

Of course, I’m old enough to remember when we had to haul the individual bits that made up our programs an hour uphill in a raging snowstorm and put them together with white glue… Seriously, the tools get better and better each year.

When a programmer’s writing or changing code, she or he is actually editing a text file. So the tool is at some level a text editor; but it’s not like anything you’d use to write a sonnet or budget overview. There’s no boldface or underlining; but there is obsessive attention to indenting; the lines of a computer program are always indented in a way that makes them easier to understand. For this to work, all computer programs are displayed in monospace fonts, where “W” and “i” are the same width; which would look klunky in your budget overview.

There are lots of these tools, each with their pluses, minuses, bugs, and idiosyncracies. People get fixated on the ones they like and powerfully emotional on the subject. But once you’ve mastered a couple, they all start to feel more alike than different.

Open source! · All the software we’re talking about here is, which means that all those pieces I mentioned are posted on the Internet and anyone can copy them and fool with them.

These days, a lot of the interesting Open Source code is posted at a site called GitHub.

It’s not the first or only public code-sharing site, but it came along at just the right time with a particularly smooth setup and rich toolset and has pretty well swept all before it. OpenKeychain is one of the many apps at GitHub.

So, when I wanted to contribute to it, here’s what I had to do:

1. Find its home on GitHub.

2. “Fork” it; that is, make a copy of the whole thing in my own GitHub space; then copy all the pieces onto my own computer. This sounds complicated but GitHub makes it easy, just press a few buttons.

3. Figure out how the pieces fit together, make the changes I want, and test them to make sure they work.

4. Send what’s called a “pull request” to the OpenKeychain team, saying “Here are some changes, want them?”

5. Assuming the team is interested at all (they were), they look at my request, probably ask for some changes to my changes (they did), and if all goes well, pull as requested, and my changes end up in the next release of the app.

6. Yay! I think I speak for just about every programmer when I say that you get this little wave of happiness when you get the email saying your request has been pulled.

Here’s what’s important: At no point in this process did I have to send anyone money or ask anyone’s permission.

And I should mention Step 7, too: After I did this work, someone else came along and figured out a better way to do a couple of the things I’d done, and requested another pull to improve my code, and the project accepted that too. (And I’ve had a couple more changes pulled since then, fixing up this and that.) This is a good thing.

Bumps in the road · The experience wasn’t quite as smooth as this might sound. I went looking for a project to contribute to and OpenKeychain was my second choice. My first choice was also Open-Source and on GitHub, but had an insanely-complex structure and, despite repeated attempts, I could never manage to combine all the pieces into a working APK. So I gave up and moved on.

Not all software is created equal. In particular, not all software is as easy to understand as other software. OpenKeychain is, based on my decades of experience, way cleaner than average and really pretty easy to figure out; a tribute to its creators.

Culture matters too; the OpenKeychain people are open-minded, welcoming, and reasonable; lots of Open-Source teams aren’t.

Also, it mattered that both the projects I was dealing with were young-ish, and not huge consumer successes (yet). If you want a change to Linux or another crucial piece of infrastructure and just march in and fire a pull request at them, you’re apt to be silently ignored.

The underbelly · The practitioners of Open-Source culture make strong claims to openness in general and in meritocracy in particular; it isn’t supposed to matter who you are or where you come from, just whether your code makes the grade.

But maybe not; others argue that the Open-Source community is a rabble of overprivileged males with an exclusion problem, a testosterone problem, a drinking problem, and a conceitedness problem; that the culture is diseased in important ways. You can even argue that the whole Open-Source thing, where people do the sort of thing I’m describing without getting paid just for love of the work, excludes people who don’t have the time, don’t have the money, or who care about life broadly enough to exclude a maniacal focus on code and coding.

And yeah, I have to acknowledge that the people who populate this culture are overwhelmingly male and white/Chinese/Indian and first-world and financially-unstressed. For example, me.

If you want to get a feel for the people who have issues with Open-Source culture, a good place to start would be at Model View Culture. I’m not gonna do any refereeing on that debate; I’m too close to it, and too conscious that my voice is awfully privileged; to the extent that I could easily drown out others less easy to hear.

But I’m not gonna stop doing Open Source work.

What actually happens · The changes I needed to make were easy, because OpenKeychain already had software to look up keys in another online directory. The mechanics were different but the basic structure was the same. There were three chunks of Java code to do all the housekeeping so that people could type in searches and see the results in a list and pick from the list to import keys. Also, a teeny chunk of XML to describe the search-for-keys button.

So I made copies of the three chunks of Java and one chunk of XML, and except for one of them, it was just a matter of changing some names.

The Java file that actually sent queries to Keybase and dealt with the results had to be pretty well written from scratch, but it wasn’t hard, because the Keybase people have gone to some trouble to make this easy. Here’s a link to it, and here’s a snapshot of a little piece.

I wrote that! Remember what I said about obsessive indentation?

And finally, I noticed that extracting the data I needed from the messages coming from Keybase required writing the same code over and over, so I created an entirely new Java file to automate that, which was actually interesting enough to write a geeky blog post about.

I ended up increasing the number of files that go into making OpenKeychain by 5.

Swearing out loud · That last piece of code I mentioned had a real actual error in it, something that caused the dreaded “Unfortunately your app has stopped” message we all see from time to time on our mobile devices. It was a sneaky one, it took a half hour of debugging. Which is something I can’t do without a lot of filthy out-loud swearing, first at the bug for being hard to find, then at myself for being stupid; in this case, I’d subtracted 1 from a number that needed 1 subtracted from it, but in two different places.

There are a lot of ways to debug things and the programs-to-write-programs inventory includes debuggers that let you peek into the code as it runs. I don’t use those much. I (and many other programmers) prefer an old-school approach where we insert what are called “print statements” in our code, to print out the values we’re manipulating as we go along. It turns out that when you have your Android plugged into your computer, there’s a special command that lets you see all the stuff all the programs on the Android are printing out. It’s sort of shocking the first time you run this, because lots of them are quite chattery.

Anyhow, when debugging, life is like this:

1. Put some print statements in your program.

2. Wait 30 seconds while your APK builds.

3. Install your APK on your mobile.

4. Pick up your mobile, run the app, look at the print statements on your computer screen. If you haven’t fixed the bug yet, your app crashes.

5. Think about what the printouts told you about the internal state of your program. Make some changes. Go back to step 1.

This is typical · There are a lot of different flavors of programming. Some people grind away making game monsters scarier and shinier. Others build wonderful new applications from scratch, behind closed doors. Others work for contractors on some corner of healthcare.gov or a private-sector equivalent.

But the activity I describe here is awfully damn typical of what a lot of programmers actually do, day by day. In particular:

• The code didn’t really make anything new, it just integrated one existing piece of software with another.

• Everything involved was Open Source.

• All the software relied on huge swathes of pre-existing Open-Source libraries written by others, sometimes years or decades ago.

• The actual work involved twiddling Java and XML files.

• The actual work was done using a big complicated software tool built to help programmers.

• The software was hosted on GitHub, and the change was accomplished by forking a project and sending a pull request.

• The whole thing was profoundly collaborative.

Overlooking the landscape · I hope a picture is emerging in your mind: A vast landscape of software out there, mountains and valleys and forests and oceans of it, assembled in big and little pieces by thousands of people over years and decades; almost all the individual contributions are undetectably small, measured against the magnitude of the whole.

We build great things; in small pieces, and working together. We have technology and culture problems; but that’s part of the human condition. I like being part of the process.

---

---

Contributions

---