I’d been going to do a REST news round-up, but Dave Johnson beat me to it. He calls it “Atom and REST” and yes, I’ve noticed a pattern where you get a handful or more of RESTafarians in a room, virtual or real, and Atom keeps bubbling up. Not just the protocol, but the data format too. Oops, Dave missed one: Pete Lacey.
On a related REST subject, Aristotle Pagaltzis, Stefan Tilkov, and Mike Herrick have been talking over whether RESTful services need a description language in general or WADL in particular.
I usually agree with Aristotle, but I think the answer is obvious. Enough people think that you need to describe services formally that not having one is an unnecessary barrier to adoption. Personally, I’m one of those people. I think writing formal schemas is a good idea for language designers even if they never get used at run-time; they’re good for documentation, tooling, and (not least important) forcing you to think formally about your problem space.
I like WADL but who knows, maybe it’s not the last word. Either way; yes, there is a hole in the ecosystem here.
Contributions
Comment feed for ongoing:
From: Jay Carlson (May 28 2007, at 23:29)
No, there really isn't a hole in the ecosystem. WSDL's primary bid for life is that given an RPC service, you can provide a summary of its parameters such that it's cheap for some vendor to spit out a cheap way of RPCing it from clients.
REST is not RPC.
A consumer of a REST service basically has to deal with all of the annoyances of an ordinary web site, from "I've moved to a different place" to "slashdot linked to me and I can't cope" to "wtf is this mime-type, I told you in my Accept header I don't do this any more". The reason the Web works at all is that it is deeply, fundamentally flaky.
The only reason to have a RESTy analog of WSDL is if it will save anyone any significant time. And I'm *strongly* inclined to think that it won't, especially when thinking about both producer and consumer time. Since the dominant mode of the REST community is "make it brutally simple", it's likely that all of the interesting constraints are going to be captured in prose descriptions rather than signatures.
By all means, pave the cow-paths, but I'd like to see them first.
[link]
From: Asbjørn Ulsberg (May 29 2007, at 03:08)
I agree and think WADL, on the functionality level, hits the spot. I would like to see an Relax-NG compact syntax for it, though, so it would be more finger-friendly to type in. We also need a lot of tool support (from Microsoft in particular) to make it as easy to use as WSDL is now.
[link]
From: Rami Kayyali (May 29 2007, at 08:06)
I also believe that some formality is required to encourage adoption of new technologies, but WADL? Don't you think that this is exactly how WS-* went downhill? They started with simple specifications and kept complicating things more and more until WS-* became nearly useless.
[link]
From: Andrew Wahbe (May 29 2007, at 08:08)
There is a documented API in a RESTful system, but it's in the client, not in the server. The client exposes an API that is driven by hypermedia. Some of the functions it exposes tell it to follow hyperlinks (and govern the method used). By here follow means make the request and treat the response as the new set of instructions (ie. the new state). Other functions tell it to make HTTP requests that don't result in new instructions but change the state of the resources on the server (i.e. ajax requests) or transfer supplemental data to the server (i.e. links to images). Other functions don't involve HTTP requests at all.
The hole in the existing infrastructure is that we don't have a good hypermedia format for machine to machine interactions. We need a client API or perhaps a better framework for building client APIs. We don't need a mechanism for describing the Server API -- it's implicit in the data we pass to the client API.
[link]
From: Gary Bernhardt (May 29 2007, at 08:52)
It seems like WADL is an attempt to pound the RESTful web into a shape that's recognizable by the RPCful brains most of us still carry around. In RPC, it's essential to have an out-of-band description of the service, because the RPC functions are not self-describing. In REST, the resource representations *are* self-describing, so you should never need an external guide to figure out what you can do next.
A RESTful resource will have links to other resources, and should tell you which verbs you can use on them to achieve different results. So you have things like this pseudomarkup:
"""
My name is Bob
I live at /users/Bob
To view my forum posts, GET /users/Bob/posts
To send me a private message, POST to /users/Bob/messages with media type "usermessage"
To update my user details, PUT me to /users/Bob with media type "user"
"""
When you GET the Bob resource, the HTTP Content-Type header will already have told you the media type. Assuming you understand that media type, you can look at the representation and figure out what links it has, what methods those links support, and what media types they want. This is basically the job that WADL's doing, except WADL does it by stepping outside of REST, whereas a solution like the one above solves the problem in a RESTful way.
In a RESTful system, the only knowledge that needs to be shared between the server and client is the structure of the media types (not the structure of the URLs!) This is a job for a schema language, not a service description language.
[link]
From: Michael Daines (May 29 2007, at 10:55)
I think I agree with Andrew... What about ActiveResource? It assumes/enforces a standard way of exposing resources, and if you co-operate you get a client API you can play with in just a few lines.
[link]
From: John Heintz (May 29 2007, at 11:46)
I agree with Andrew Wahbe on this.
WADL is better than WSDL, but fundamentally the same thing: an up-front programming contract.
There is a hole in the eco-system: a true challenge today trying to write machine driven RESTful clients. Jumping to a contract-first solution (because it is obviously comfortable) is not going to work any better for multi-organization B2B than WSDL though.
This style of specification ignore two key characteristics of REST:
1) Hypermedia as engine of application state, and
2) Mobile code (HTML Forms and JavaScript today).
Turning speculation on...
I don't have an answer, but instead of specifying and templating the URIs, a focus on data and media types might be better for this.
If media types (or micro-formats) existed for:
* categories (bread crumb trail, children, related)
* parts (part name, category, id number, cost, related parts links, specification link to name-value pairs)
* shopping carts (add to cart form, shopping cart link, remove item form, purchase link, mailing address, credit card info, confirm)
I could write agent code against those formats that could shop for a specific camera or any camera <$400 with "megapixels" > 7.
Point the agent at Amazon and Shop.com and let it decide which one has the better deal.
No link templates, no programming contract. Just an agreement on the data formats and a client that can decide how to interpret that format and follow links from the server.
...end of speculation and long-winded rant.
John Heintz
[link]
From: Paul Downey (May 29 2007, at 13:34)
I'm a bit thick, so bear with me: what's the difference between a WADL document telling you how to construct a few URIs and what to you might expect in return and a HTML form telling me how to construct URIs and what's likey happen as a consiquence, especially if both docuemnts are "on the web"?
[link]
From: Joe Pallas (May 29 2007, at 14:22)
I understand why some people are nervous about WADL: it has a faint whiff of the "self-description spontaneous organization fantasy" about it. (This is the amusing notion that if a bunch of services are self-describing, and they are made aware of each other, they will somehow magically turn themselves into an application -- despite having never heard of each other at the time they were built. EJB and WSDL both seem to be based on this fantasy.) When I see a machine-readable description format, I get suspicious about what people expect it to do for them. But a human-readable description is necessary, and formalism is good. For better or worse, we tend to use XML for our formal descriptions in this community, so there we are.
The people who argue that REST-style architecture obviates the need for any sort of external description are, I think, living in almost the same self-description fantasy. Sure, I can retrieve a resource given a URI and the resource will have a MIME type along with it. But unless that's a MIME type I know something about, I'm not going to manage to do anything useful with the content. Nor does it do me any good to know that I can post to a given URI, unless I also know what will happen when I do so.
If Atom is a paragon of REST-style applications, people should notice how much of the Atom spec is not devoted to schemas or when to use PUT vs POST. There's a lot more to a useful protocol than syntax.
[link]
From: Joe (May 29 2007, at 15:42)
If you write a spec, write a validator alongside. How much pain could have been spared with early versions of RSS if we'd had a common, agreed upon validator.
In short, it's the test suite that ultimately decides the spec. I agree, spec's are an incredibly good idea - and too often nearly useless in english (or other human languages).
[link]
From: Mark (May 29 2007, at 19:45)
The wonderful thing about RSS validators is that there are so many of them to choose from.
Joe is wrong; there was an RSS validator early on. Not Netscape-era, but certainly early-Userland-era. It lived at http://aggregator.userland.com/validator . There were only two problems with it:
1. It was closed-source.
2. It was crap.
( Discussion of both points: http://diveintomark.org/archives/2003/06/26/dont_use_userlands_validator )
It was shut down ages ago, replaced by a hostile (and ultimately unmaintained) fork of the validator which Sam Ruby and I wrote in the meantime (and which Sam still maintains).
Recently I've heard murmurs about another group forking our validator *again*, for reasons unknown and likely unfathomable.
So yeah, build a spec, build a validator. But don't think that's the end of the story. Often, it's only the beginning.
[link]
From: Chimezie Ogbuji (May 29 2007, at 19:59)
RDF Forms seems to hit the right spot with a minal description of a REST service (protocol semantics).
[link]
From: pwb (May 29 2007, at 22:24)
I'm not sure we need a hard contract and validation but we sure as heck need a tiny bit of guidance on how to design RESTful interfaces.
A problem I see with XML is that it seems to lead to overly complex data structures. Most APIs would be fine with 2D or max, 3D. At the end of the day, data usually needs to be representable in rows/columns so it's not totally helpful for data to be much more complex.
[link]
From: Jay Carlson (May 30 2007, at 01:50)
HTML forms are really the key to a lot of this.
I've got a generic text-processing kind of service I care about a lot, and I'm pretty sure that every single vendor has managed to find some kind of new and interesting way to talk to it---everything from "here's a DLL, good luck" to "here's a SOAP interface."
OK, so I model the services as URLs that accept text/plains and return text/plains. But every vendor has their own knobs and dials that a user might improve the fidelity of the service. Like, when you generate Japanese text, what level of politeness do you want?
We're totally out of the realm of what RPC can handle by itself here. I mean, assuming you can do machine-to-machine discovery of parameters, what exactly do you expect a search engine to say about the lexical stance of texts it processes?
Eventually you're going to need a user interface to choose these kinds of things. Only users can really understand the tradeoffs. And at that point, the best thing an underlying service can do is serve up an HTML form explaining "here are all my knobs and their explanations; set them how you will, and I'll grab their query-encoding when you hit OK."
[link]
From: John Heintz (May 30 2007, at 10:50)
Paul Downye: The difference is that a WADL spec is used to generate code at design time, and an HTML Form is used to generate a URI at runtime.
I completely agree that we need *something* to be useful at design time, but enforcing URI space at design time is not RESTful. A RESTful server should be able to change the URI generation scheme without breaking clients. (Bookmarks are still a problem, this isn't a single answer type of problem).
Joe Pallas: Coding the protocol into the client is counter to "hypermedia as engine of application state". Instead the data (hypermedia) exposes the protocol steps available as links. Both client and server need to understand the meaning of the data, but the protocol (e.g. can't confirm order before adding items to cart) is exposed over time.
This "exposed over time" bit is the challenge for machines. Trying to nail that down (up-front spec like WADL) is not a RESTful solution, and will always be a challenge when any one of the systems upgrades to a new version. See the discussions on XSD versioning for problems.
To extend my example from before:
Today: An online store has a single web form to class="shopping-purchase" a shopping cart of items. This single form has all of the input fields for mailing and billing address, credit card info, and so on in one form.
Tomorrow: The online store has a three-step form (still each one class="shopping-purchase") that first collects address info, then credit card, then confirms.
The agent that just follows data values would (or at least easily could) work the same against both of those. If the agent wanted to buy stuff, it would find a form with class="shopping-purchase" and fill in whatever values it new about. It would continue doing that, two more times, until is got a success or failure response.
A WADL generated client would fail when the site made a change like this.
[link]
From: Wilhelm Reuch (Jun 01 2007, at 00:25)
If we already have SOAP/WSDL and even WS-*. Why would you need to do the same with REST? When those people can just use SOAP/WSDL.
And why create new description formats when we already have XML Schema? Just because people love creating description standards?
Its becoming more and more apparent that REST is going down the same slope that killed SOAP. I am going back to XML-RPC now.
[link]
From: John Heintz (Jun 01 2007, at 14:10)
I've collected the ideas in my comments here into a blog post and named the idea signposts. Anyone interested please follow http://johnheintz.blogspot.com/2007/05/does-rest-need-dl.html
[link]
From: Jacek (Jun 01 2007, at 16:24)
a reply to John Heintz, but his blog doesn't allow anonymous comments, I don't wanna sign up with blogger or google, and my blog is temporarily out of order...
It seems to me that John, in his eagerness to use microformats, reinvents the idea of a standardized structured format. We had that with SGML, we have that with XML. You don't need to use XML Schema to work with XML. In fact, (and Tim may confirm or deny this), XML seems like it's intended to be usable for these signposts John speaks of - recognized names would be my interpretation. Granted, microformats have their advantages and disadvantages over XML.
A good IDL should embrace versioning, but apart from that, I believe it's folly to dismiss IDLs - without AI (working on it...), machines still need to be preprogrammed to handle known data, and I see no use in pretending that a Web app does not have a formally describable interface that can be used by a machine.
We're just trying to find how to describe the Web interfaces so that it's not that constraining in face of common Web forms of versioning - new URIs for instance. I'm still not convinced that WADL is it, but it's intriguing.
[link]
From: Jacek (Jun 02 2007, at 05:08)
Hi Tim, I'm reminded that I wanted to comment that you could add the "contribute a comment" link at the end of the page after the comments as well, because some readers go through all the comments before writing their own, and you're making them (us) scroll to somewhere in the page where your content ends and comment begin. It's not hard, but it could be much easier to just click down there. 8-)
[link]