I was asked to help out with work on a bunch of Cloudy APIs and naturally I’m trying to make them RESTful. It dawns on me that, very often, client-to-server messages can be expressed as name-value pairs, and that doing this is on balance A Good Thing.
In particular I was seeing, in HTTP exchanges, a lot of client-to-server messages (POSTs and PUTs) that looked something like this:
<node>
<time-to-live>300</time-to-live>
<threshold>88.5</threshold>
</node>I saw almost no instances of nested structures, long-running text or the
other things that scream “XML!” My first thought was “JSON, maybe?” but, you
know, that’s maybe overkill too, why not just do name-value pairs like an HTML
form, i.e. application/x-www-form-urlencoded (Near as I can tell,
the normative reference is
from
the HTML spec; gack.)
Note that in the above example, the resource I’m sending this to represents a Node, so one layer of XML markup is superfluous.
Pro · Why would you do this, rather than than use XML or JSON? Well, because then you can get at your API through a Web browser. All you have to do is program the server to send HTML to browsers and XML (or JSON) to non-browser clients, which is pretty easy with modern frameworks like Rails. Maybe I’m missing something, but this seems like a huge advantage.
Con · I can see two problems. First, you are quite likely to feel internationalization pain; XML may not be perfect but it does make sure that client code builders have to think through the i18n. There are almost certainly going to be certain combinations of browser releases and form designs that will screw up some character encodings.
Second, browsers really only do GET and POST, and the U and D parts of CRUD map nicely in the natural REST idiom to PUT and DELETE, which are usefully idempotent. In fact, I still have bruises from batterings administered by Joe Gregorio, during the design of AtomPub, to anyone who felt that maybe the use of PUT and DELETE was not totally essential.
So, granted. But maybe Joe needs a bigger club, because I have to admit that limiting myself to GET and POST just doesn’t cause me that much heartburn.
Everybody Else Knows This Already · At least I suspect so. The years I spent grinding away on AtomPub, which is all about CRUD on documents and blobs, left me with blinkers.
Anyhow, on balance, I’m increasingly inclined to believe that there is a large class of simple administrative-CRUD apps where you can send those name-value pairs upstream, and the advantages may make it worthwhile living with both fighting the i18n fires and a certain loss of RESTafarian purity.
Contributions
Comment feed for ongoing:
From: Patrick Quinn-Graham (Jan 29 2009, at 14:33)
I did this with one of my iPhone apps - it works particularly well as parsing form key-value pairs is something anything that does CGI copes with.
You can even get "simple" nesting by using something like obj[key]=value&obj[key2]=value&secondobj[key]=... which PHP and Rails (at the very least) turn in to arrays for you. It feels a bit messy to do that though.
[link]
From: John Cowan (Jan 29 2009, at 14:44)
XMLHttpRequest operations are perfectly capable of PUTs and DELETEs, and in environments that won't let anything through but GET and POST, the X-HTTP-Method-Override: PUT header (and similarly for DELETE) is your friend.
But as for x-www-blahblahblah, its i18n story is indeed lousy, and IMHO it is worth having the slightly greater overhead for JSON: okay, so you have to use a { and } around the whole thing, and quotes around the names and values.
[link]
From: Eric Larson (Jan 29 2009, at 14:46)
This pattern is actually really nice. I made a similar system for allowing easy uploading of files at my job that allowed saving key value pairs from the multipart/form-data content as meta-data. My system was a very simple filestore backed by a couchdb-ish system. The whole system worked very nicely and easily allowed using the web browser as a client. It was also trivial to work around limitations in the browser implemented HTTP verbs with the _method practice found in most frameworks.
My other option was to effectively proxy the requests through the server using something like Pyro (this is Python), but I think by using simple form data, the browser availability greatly simplified the client code. It also made writing an actual client object trivial.
[link]
From: Sam Pullara (Jan 29 2009, at 15:00)
This is one of the reasons I like JAX-RS so much. It makes it very easy to accept JSON or XML or even form data with the same code.
[link]
From: Thijs van der Vossen (Jan 29 2009, at 15:03)
From experience, client code builders are just as likely not to think about i18n even if you use XML.
If you want to make them think about it you'll have to a) mention explicitly in your API documentation you'll be sending and expecting UTF-8 and b) you'll have to give them a few examples that will break badly if they don't do that.
You've mentioned Rails above. A nice thing about building a vanilla REST style API with Rails it doesn't matter whether you're doing things as XML or form data because both just work.
This proved to be useful in practice because for some clients it's easier to send form data, while for others (think MS Office macros) it can be simpler to send XML.
[link]
From: David Slik (Jan 29 2009, at 15:07)
We mulled over the same issues when we designed our RESTful HTTP API for object storage interface at Bycast. Ultimately, the elegance of using header name/value pairs to specify metadata associated with a stored object as part of a PUT, and to request it back as part of a GET and HEAD outweighed the issues with character encoding and other restrictions for strict HTTP/1.1 conformance.
For queries, we did go the route of using POST combined with XML query definitions and XML query results, so there are times where both approaches make sense.
[link]
From: Mark Volkmann (Jan 29 2009, at 15:21)
How about a compromise? Something that is still XML but looks more like name-value pairs?
<node time-to-live="300" threshold="88.5"/>
[link]
From: stand (Jan 29 2009, at 15:30)
I'm also doing a similar project. It seems a lot of people are these days. I went through the same mullings over using using application/x-www-form-urlencoded (I did), and giving up on PUT and DELETE (I didn't).
I particularly like having DELETE because it means that I can just submit a application/x-www-form-urlencoded entity and the server can know to remove those key/value pairs just from the HTTP Method. If you want to use POST, you need some magic words in the content to let the server know what to do.
On a somewhat related note, Joe's httplib2 python module really kicks ass!
[link]
From: len (Jan 29 2009, at 15:33)
Deja vu. I was sitting with Newcomb in yet another hotel lobby back in the day before XML and I asked him why the resistance to markup was so daunting among the webHeads and how they planned to send documents. He said, "They have name-value pairs and they think that is all they will ever need."
So maybe XML is brain damage.
Ah so. Back to hunting for a Senior .Net programmer to hire.
[link]
From: pwb (Jan 29 2009, at 17:08)
I am ABSOLUTELY STUNNED that simple querystrings never caught on as a response format. XML and JSON are way overkill for the bulk of situations. JSON's is only useful for AJAX. No one uses JavaScript for server-to-server API calls. ANd XML is just a PITA.
It's very rare that you really need more than one dimension of data.
And here's a crazy idea for two dimensional data: HTML tables!!
I think it's perfectly fine to go with properly used POST and GET. DELETE is totally useless, you never really delete something, you just change it's status. And I don't even know which of POST/PUT is C or U. Doesn't seem to matter much.
IMO, public APIs are still way too overcomplicated.
[link]
From: Gavin (Jan 29 2009, at 20:21)
application/x-www-form-urlencoded is indeed wonderful and works very well. As for internationalization your in the same boat you are with any URL encoded system. In theory the URI should be made up of ascii characters with any other characters escaped as %HH where HH is the two byte code... heh, of course this leaves lots open for discussion. In practice assuming that URIs are escaped from UTF-8 works as specified in err.. some RFC I can't find at the moment. Anyway, the other half of application/x-www-form-urlencoded is the wonderful text/uri-list a very simple and easy way to to return a list of links. There a lot of good media types out there that seem to be ignored in favor of XML or JSON.
[link]
From: Dirkjan Ochtman (Jan 30 2009, at 01:18)
One of the advantages of JSON that I really like is that it gets you simple typing (that is, numbers, bools, strings). Easier than having sentinel values (for bools), and helps communicating intent.
[link]
From: Damian Cugley (Jan 30 2009, at 03:02)
URL-encoded character data should encoded as bytes with UTF-8 and then be encoded as a URL using %-escapes for non-safe characters. This has been the convention for donkey's years, though plenty of sites violate it.
Anyone want to try to persuade browser writers allow forms with the attribute enctype="application/json"? Less ambiguous, less trouble to encode, and less trouble to decode than x-www-urlencoded
[link]
From: Sore Foot (Jan 30 2009, at 05:39)
Consider a semantic web approach of passing rdf (triples or quads).
[link]
From: Jacek (Jan 30 2009, at 06:58)
@gavin: there are a lot of simple media types that work well until you need to extend the data sent around.
URI list - what happens when you want to add simple metadata about those URIs?
XML may be overly complicated (it did come from SGML, after all) but I for one very much like features such as namespaces, xml:ID, and the I18N support.
But Tim talks about how using x-www-form-urlencoded enables you easily just to whip out an HTML form to use your API. That's one huge benefit, IMHO.
And then your PHB comes and tells you to add API keys and a fancy-shmancy authentication, and say bye-bye to the usability of your API.
[link]
From: Peter Keane (Jan 30 2009, at 08:07)
I'll second Gavin's recommendation for text/uri-list -- I have found it quite useful, not just for server-to-client but also for things like (somewhat RPC-ish??) "go grab the resource at this url (or these urls) and save it. I simply POST the uri-list to my 'ingester' end point. I'd also put in a plug for 'text/plain' -- when the end point is granular enough to simply be a value (e.g., PUT to "http://example.com/widget/123/model-number").
Also -- I do find great benefit (cleaner, more rational interfaces) by using all of the Big 4: GET, POST, PUT, DELETE, XMLHttpRequest being the key.
[link]
From: Kevin Hutchinson (Jan 31 2009, at 15:19)
I think posting JSON is simplest, such as:
json="{bangers:1, mash:0, dinner:'ready', sauce:'hp'}"
[link]
From: pwb (Jan 31 2009, at 18:27)
That's simpler than "bangers=1&mash=0&dinner=ready&sauce=hp"?
[link]
From: Kevin Hutchinson (Feb 01 2009, at 15:24)
True, POSTing form values would be simpler than JSON - I'll share my HP sauce with you. ;-)
I was just thinking about the issues of query strings on GET requests and how they end up in browser histories and web server log files - which are too public.
[link]