Dynamic Java

It’s pretty clear that dynamic languages are a hot area, maybe the hottest, in the world of software development. We need to do more to make them easily usable by people in the Java ecosystem. So on Tuesday we held a summit here at Sun, with a few of our internal Java leaders, and on the dynamic-languages side, Larry Wall and Dan Sugalski (Perl and Parrot), Guido van Rossum, Samuele Pedroni and Sean McGrath (Python), and James Strachan (Groovy). It was an educational day for us; herewith some take-aways and pictures.

Starting Now · We’re already doing reasonably OK with dynamic languages in the Javaverse. Here is Sean McGrath’s write-up on doing servlets in Jython, and over at O’Reilly Marc Hedlund has a nice little demo of building an RSS aggregator in Groovy. Gotta love this:

class Item { title; contents; String toString() { return title; } }

But Jython and Groovy (and there are others) have been built mostly without help or encouragement from Sun, and quite likely we can make their implementors’ lives easier.

Java has a PR problem; while Microsoft marketed the .NET stuff as multi-language from day one, the fact that Java is a three-legged stool (language, JVM, libraries) kind of gets lost under the enveloping carpet of the one-word name “Java”. Partly that’s Sun’s fault; in the early days, the evangelists generally left the impression that anyone writing code in a language other than Java was a second-class citizen. Which is silly; Java’s a good language but I often like to do things in Perl and Python, and so do lots of other developers.

The Summit · Even if Sun didn’t approve of other languages on the Java platform, they’d happen anyhow. I approve, and when I started going around Sun asking, it turned out that everyone I asked did too. So I asked Graham Hamilton, who’s kind of at the centre of the Java universe, if he thought it would be a good idea to bring in a roomful of dynamic-language experts to help us figure out how Java could be made a better home. Graham did, and also pointed out that since we just shipped the 1.5 “Tiger” release and are early on in the planning phase for the next, right now would be a good time.

So I talked to Juan Carlos Soto, who’s our director here at Sun’s Software CTO office, and he thought it was a good idea too and agreed to fund it.

The Players · We had, from the Java team, Graham Hamilton, who’s a VP and Sun Fellow and lead architect for J2SE, Gilad Bracha, who handles the language and JVM specifications, and Martin Buchholz, a senior Java maintainer (and at one time XEmacs maintainer).

On the Perl side we had Larry Wall, who’s currently engaged in defining Perl 6, and Dan Sugalski, who’s writing Parrot, the new Perl VM.

From the Python camp we had Guido van Rossum, Samuele Pedroni, currently Jython lead, and Sean McGrath, whose company Propylon employs a small tribe of programmers actually building and shipping real Jython applications for real paying customers right now.

Groovy was represented by its lead James Strachan.

They gave us a teeny little conference room and it was crowded. I have never claimed to be the world’s best programmer, but it’s not very often that I’m in a room with a bunch of people talking about software issues and they start going right over my head. But that happened a couple of times, which was a problem since I was supposed to be taking notes.

For a touch of flavor, here’s just part of the list that of things to discuss that Larry had prepared: anonymous code/closures, lvalue methods/functions, variadic call/return, constant/rw/copy/ref/lazy parameters, wrappers/AOP, eval, exception handling/undef/nil/unthrown exceptions, exception handlers with lexical access, temporization/hypotheticality, efficient regex with complete semantics, access to dynamic context/want/caller, redispatch of methods (fallback upon "fail"), efficient switch statement?, versioned modules/classes, virtual classnames, continuations.

Lesson: Dynamic Dispatch · The semantics when you do a Java-to-Java method dispatch are a lot more subtle and complex than you might think, but they’re all handled by the compiler and the JVM apparatus and it’s super-efficient. Dispatching from Groovy to Jython to Java and back is a lot more complicated, and the Groovy and Jython guys have had to write tons of code to figure out which method to call and get all the arguments in order and make the dispatch happen. It’s tricky dispatching both to Java and to a dynamic language, especially since the dynamic languages tend to keep meta-classes around for the Java classes.

All this is inefficient. So it might be worthwhile to add a new bytecode to the JVM to help with at least some of this choreography and housekeeping; especially since it doesn’t look like that bytecode would be very expensive to add.

If this idea holds up after a bit more thought there’d have to be a JSR so the whole community can kick it around, but it looks plausible.

Lesson: Bytecode Generation · A lot of people these days are generating bytecode: not just Jython and Groovy, but AOP tools and XPath/XSLT tools and others too. They’re all doing it by hand without any abstractions to help, and it’s difficult and messy and they’re probably not doing that great a job.

So it would be super-handy for a lot of people if there were a higher-level approach to this, perhaps at the AST level, so you could automate the bytecode generation for some common idioms like switches and loops and so on. It’s trickier than you might think, since the semantics of something as apparently simple as assignment to a variable can get very complex in a dynamic language.

Janino looks interesting in this respect, and some of the work in Kawa might be re-usable, and over in .NET-land there is this thing called the CodeDom which is apparently addressing a similar problem. So maybe the right thing would be a java.net project to build some shared infrastructure in this space.

Odds & Ends · We had lengthy, heated, and amusing debate on closures, continuations, and tail recursion. It would be reasonable to think about adding bytecodes to support these things (although doing closures well would be very expensive), so the question is how much this would help the dynamic-language implementors; for example, Jython’s implementation of Python generators works just fine. There wasn’t a mob howling for tail recursion, although Dan Sugalski gave us some good scenarios where it really helps bring sanity to the implementation of some very natural and useful algorithms.

Going Forward · There were a couple of action items to report back on this or that, but the take-aways were pretty clear. For my money, the important thing about the meeting was bridge-building; the dynamic-language guys now know who to shout at in Javaland, and the Java guys know who in dynamic-language land to pester about what works and what doesn’t.

And I confidently predict that as time goes by, more and more of the code written in the Java ecosystem won’t be in Java. This is a good thing.

---

---