I managed to ignore Atom for a few hours this week and get back to working on project Zeppelin, which leads to a few thoughts on object transmission, concurrency, Jython, and other stuff of possible interest to hands-on Javans.
Writing Objects Fast ·
A few weeks back, I
wrote about improving the
latency of client-server
Java Object network transmission from appalling to merely bad.
I had trouble getting it to run any better than bad, then
Mike Duigou made the
obvious suggestion that I jam a BufferedOutputStream in front of
the socket, d’oh, now
I can push a thousand objects a second through a decent network link. ¶
The difference between this time and all the other times in my career I’ve made systems run faster by buffering output is, I didn’t have to write the software myself. This “Object-Oriented” stuff is gonna catch on. ¶
Concurrent Pain · The way Zeppelin works is you’ve got a client that talks to a bunch of different servers, and in some cases it has to send a transaction to all of them. Stress-testing showed me the obvious, that doing this one by one was slow and stupid, so each client now keeps a bunch of pet threads around and when a transaction needs to go everywhere, you load it up and unleash all the threads, which write and read and report back, and you’re done. The efficiency went up enormously. ¶
Once I’d debugged it, that is.
Concurrency, well y’know, it’s hard. After you’ve finished debugging it,
it’s obvious that if the parent says while (true) { load(transaction);
start(pets); waitfor(pets); } and the pet threads say
while (true) { waitfor(parent); send(); receive(); if(iAmTheLastPetToFinish)
{ tell(parent); } }, there’s this hole in the concurrency logic that you
only fall into on about the eight thousandth time around the loop, varying
wildly as a function of anything else the computer might be doing. ¶
The first problem is that old programmers like me have a bunch of well-debugged design patterns in our heads which get us through most problems without thinking too much. But we don’t (well, I don’t anyhow) have nearly enough concurrent-system design patterns in there. Is there a book someone would like to recommend? ¶
The second problem is that I haven’t figured out how to do Test-Driven Development of concurrent systems. I know how to write tests that catch an iterator that doesn’t iterate and numeric code that loses precision and a million other stupid things that every programmer does. But I don’t really know how to write tests that catch simple, bone-headed, obvious errors in concurrent systems. So I end up using a lot of print statements. ¶
Dynamic Languages, Please · Another routine in the grotty underbelly of Zeppelin does a bunch of file/directory maintenance: create a shadow directory tree if it’s not there and copy a bunch of files selected by extension hither and yon. I’m sure programmers will still be writing this kind of code when I’m a hundred years in the grave. ¶
Well, Java is really not the right tool. What would have been a few lines of straightforward Perl turned into half a dozen klunky-looking Java methods. If I’d been able to rely on Jython just being there and in my IDE, I would have come out way ahead. Fortunately, I think we’re going to fix that one... ¶
