Here it is Friday afternoon, and my efforts to lash my brain into actual technology creation are becoming less and less effective. So let’s dabble in meta-REST instead, shall we?
On Wednesday we unveiled the first public draft of the APIs for the Sun Cloud. There was a ton of feedback, in comments, twitter, email, and the project forums.
One of the comments wondered why, to create things, for example a VM in a
Cluster, you needed to POST to a special create-vm URI. Why
not just POST the representation of the VM to the cluster?
Uh, well, because when we cooked up the idea of special purpose “controller” URIs, we liked it so much that we went overboard. Unless someone has a good reason not to, I’m going to go and make the obvious changes: To create a Cluster or an Address or a VNet, POST the right representation to a VDC’s URI. To create a VM, POST to a cluster’s URI.
Which, by the way, will require almost no code changes.
The next argument is about all the other “controller” functions. Deploying
a model, starting and stopping and rebooting a machine, attaching networks.
The argument is that it’d be more RESTful to have some state
fields in the appropriate representations, and just update those fields to the
desired new state values.
Now, doing this, once again, would require almost no code changes.
But I don’t buy it, and here’s why. If I want to update some fields in an existing resource, I’m inclined to think about PUT. But that doesn’t work because it’s supposed to be idempotent, and rebooting a server sure isn’t. Well, OK, do it with POST I guess; no biggie.
But you’re not really changing a state, you’re
requesting a specific set of actions to happen, as a result of which the state
may or may not attain the desired value. In fact, when you hit the
deploy switch, the state
changes to deploying and then after some unpredictable amount
of time to deployed. And the reboot operation
is the classic case of a box with a big red switch on the side; the problem is
how to push the switch.
So, the more I think of it, the more I think that these resources are like
buttons, with only one defined operation: push. People have been whining
about “write-only resources” but I don’t have a problem with that because it
seems accurate. The
reboot and halt buttons don’t really have any state,
so you shouldn’t expect anything useful from a GET.
And finally: as I’ve said a couple of times now, moving from one of these models to another has almost no effect on the underlying code. REST isn’t good because it’s REST, it’s good because it’s good. Which is to say, I’d like to hear an actual engineering/implementation argument as to why POSTing state is better than just hitting the switch.
(Related: Bill de hÓra’s “Just” use POST.)
(Related: Roy Fielding’s It is okay to use POST.)
Contributions
Comment feed for ongoing:
From: John Cowan (Mar 20 2009, at 16:29)
The REST way would be to expose readable machine state such as Up, Down, and Crashed, and then allow you to change the state with a PUT. That doubles the latency on a reboot, but hopefully you don't need to reboot often enough to make that a big issue. For deployment, the state would be Undeployed, Deploying, Deployed, and that would allow you to undeploy as well as deploy with an appropriate PUT.
[link]
From: Tim (Mar 20 2009, at 16:42)
Uh, John, no. PUT is required to be idempotent, and rebooting a computer isn't.
[link]
From: Seth Ladd (Mar 20 2009, at 16:47)
I'm glad you brought this up, because it's interesting and I think we can learn a lot from practical implementations. Here's my take on it.
I've always thought that, instead of thinking about state (which often implies there is some attribute with a particular value), it's more productive to think about Nouns and Verbs.
Now, it's easy to argue that in HTTP the verbs are the HTTP methods. This maps well from theory to implementation, and is easy to map REST to HTTP this way.
The trickier part is to learn how to model the Nouns in the system. Nouns are the "things" (aka entities aka Resources) that we're modeling and affecting via the verbs. It's these Nouns that encapsulate state.
Let's think about running for a minute. If I wanted to model Running, I wouldn't have a Person noun with a property called "state" with a value called "running". First off, there's lots of different states that person could be in. Second, it's hard to talk about this run (as a thing). When did the run start? When did it finish? How far? If you are modeling state as a value of some property, you lose the ability to talk about and add metadata. (yes, you could make the value of the property an object itself, which is what we're leading to)
If I wanted to model Running with REST, and I wanted to start a run, I wouldn't PUT the representation of a Person with his "state" set to "running". Instead, I would want to POST a new Run Noun to the collection of all activities (or more specifically, the collection of Runs)
POST /people/12/activities/running
{
type: 'Run',
start: 'now'
}
I have finished the run? Just PUT to the URI returned when first POST the run.
PUT /people/12/activities/running/554
{
type: 'Run',
start: 'then',
stop: 'now'
}
We've turned Running from a simple state of a person to a true Noun which can be created, updated, and talked about.
Taking this back to rebooting machines. I would argue that you would POST to /vdc/434/cluster/4894/server/4343/reboots Once you've posted, you have a URI which represents *this* reboot, and you can GET it for status updates. Through the magic of hyperlinking, the representation of the Reboot is linked to the Server that is rebooted.
Hope that helps. I think minting URI space is cheap, and URI's are even cheaper. Create a collection of activities, modeled as Nouns, and POST, PUT, and DELETE away!
[link]
From: Chris Quenelle (Mar 20 2009, at 17:04)
The data types for "computer maintenance commands" and "computer active state" are different types with different values. It's easy to conflate them if you treat them too simplistically.
[link]
From: Benjamin Black (Mar 20 2009, at 17:54)
You can either use POST or define the rebooting states and transitions such that reboot is idempotent.
bb
ps - Please consider giving credit to folks whose feedback you implement. I'm not in the Zen-like state @swardley manages, yet.
[link]
From: Peter Keane (Mar 20 2009, at 18:24)
Seems to me that a reboot is not really an appropriately decomposed task for a REST architecture. I could see a "stopped" state and a "running" state, both suitably idempotent. A reboot simply means setting (PUT) state to "stopped" (and repeat if necessary 'till you get a 200) and then setting state to running (again, as main times as you wish until you get the 200). Whether you really want to do that or not, I don't know. Of course you could "compose" a POSTable end point that did it (no reason the server can't be it's own client for this purpose) and at least part of the interaction is pure REST. So you get some fine-grained idempotent capabilities ready for reuse...
[link]
From: Jeff Hodges (Mar 20 2009, at 19:00)
You could keep a random variable in the db that acts as a guard. The random variable gets changed when the machine receives a message to reboot or shutdown or whatever. PUT requests to reboot the machine check that random variable is the same as the one in the db. If it isn't, the request is ignore. If it is, the machine is rebooted, the variable in the db is changed and life keeps on its way.
Now, you'd have to wrap this db write in a transaction, but if you're keeping track of the machines in a db already, you probably should have one there already.
Basically, this is a reimplementation of CouchDB's versioning system.
[link]
From: Mike Amundsen (Mar 20 2009, at 19:05)
Sounds like the resource is the control state of the VM.
GET /vms/33333/control-state
Response:
200 OK
{
"name" : "web01",
"uri" : "http://example.com/vms/33333"
"state" : "STARTED",
}
-------------------------------
PUT /vms/33333/control-state
{
"run-status" : "HIBERNATE",
}
Response
200 OK
{
"name" : "web01",
"uri" : "http://example.com/vms/33333"
"state" : "STARTED",
}
a later GET /vms/33333/control-state
might return
200 OK
{
"name" : "web01",
"uri" : "http://example.com/vms/33333"
"state" : "SLEEPING",
}
and so forth.
you might also expose a resource that returns the control-state of a collection of vms, etc.
[link]
From: Rob Sayre (Mar 20 2009, at 19:38)
Consult RFC 2324, section 2.1.1
[link]
From: Andrew Wahbe (Mar 20 2009, at 21:10)
Worrying about the way you've structured the resources on the server is usually a sign that you actually aren't thinking RESTfully. Why? Because you should be worrying if your hypermedia format frees the client from any dependencies on that structure. If you've got that right (and most haven't) then you can worry about using the method with the strongest semantics for each operation (GET for idempotent and safe operations, PUT or DELETE for idempotent and un-safe operations and POST for everything else). That will allow you to tune the plumbing (e.g. caching) as needed. Anything else is a waste of time. Futzing with resource structures doesn't yield practical benefits and doesn't make your system more RESTful (despite popular opinion).
[link]
From: Ed Davies (Mar 21 2009, at 04:34)
This sentence:
"If I want to update some fields in an existing resource, I’m inclined to think about PUT."
doesn't seem to fit with the rest of what you are saying and could be very confusing if taken out of context. I suspect you are thinking about PUT on a "sub-resource" as PUT should be reserved for complete replacement of the addressed resource? RFC2324 section 9.6:
"If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server."
It's a "modified version", not a "modification to the version".
[link]
From: Steve Loughran (Mar 21 2009, at 05:05)
Assume every entity in the entire deployed cloud application -vms, filesystems, hadoop namenode process, whatever, has a URL and a state attribute. If you PUT it into the running state and it is already in running/live then no effect: you are putting it into a state it is already in. If it is in the failed state and you want to put it into started, well, that depends on your lifecycle. Do you allow restarts, or do you destroy that resource and create a new one at the same URL? As for reboot, that's a sequence, maybe a verb, but one that is a shortcut for a PUT to stopped and a PUT to started
To be honest, it's not much of a detail. Post works too, and goes well into forms. But it is interesting to see why the decisions have been made.
[link]
From: Mike Kelly (Mar 21 2009, at 08:04)
The problem I see here is that ’switching’ to reboot does, infact, seem like an idempotent action - the idea being that a ‘rebooting’ state would actually be a graceful shutdown, to an off state, and back to an on state immediately. Once this rebooting state was initiated any further updates to the rebooting state would be idempotent:
PUT /vm/test-machine/power
{ ’state’: ‘rebooting’ }
Maybe the kind of reboot you're talking about is in fact a quick switching of states to off and then on again (i.e. non-graceful rebooting).
That still doesn’t need a POST - it needs 2 PUT’s on the power state; from on to off, and from off to on again.
GET /vm/test-machine/power
{ ’state’: ‘on’ }
PUT /vm/test-machine/power
{ ’state’: ‘off’ }
PUT /vm/test-machine/power
{ ’state’: ‘on’ }
* once again both of those PUT actions are idempotent, and seem to be more consistent with the nature of the HTTP verbs available for this
[link]
From: Erik Mogensen (Mar 22 2009, at 23:23)
@Tim and @John, I always believed the way to get PUT to be really idempotent was to include conditional headers. This is my contribution to the debate.
GET /vm/state
ETag: "state-1"
... "on" ... (or any other myriad of examples from this comment's contributions)
To reboot (or power off or whatever:
PUT /vm/state
If-Match: "state-1"
... "rebooting" ...
Now if you're not sure that the message went through because a router blipped or a proxy was down (and you got some 5xx error) you can safely reissue the exact same message and nothing bad will happen, since the preconditions now fail.
[link]
From: Michael Richardson (Mar 23 2009, at 08:21)
Wow, Seth got it entirely right.
Not only does this make it entirely idempotent, but YOU REALLY WANT the audit log of having rebooted the system in the system.
WOW. Gotta say it again.
[link]
From: Roger Wallace (Mar 24 2009, at 03:50)
When you "reset" or "reboot" a service, server or device you are temporarily removing that resource, with the hope that it comes back up at some point. It might make sense to DELETE the resource and then refresh its main/state URL with a GET.
[link]
From: Paul C. Bryan (Mar 24 2009, at 08:29)
PUT can't be idempotent unless it is made conditional (e.g. If-Unmodified-Since) or qualified (e.g. version in address, which cannot be modified more than once: PUT /resource/12, version is incremented).
One should be able to PUT the resource with a state of "reboot" if the state of the resource is known to be a particular state at the time the PUT occurs, be it a last-modified time or resource version.
POST seems fine too; perhaps not as elegant as conditional PUT, but if your consumer doesn't want to know the state of the resource -- or can't -- prior to the request, POST seems like the only reasonable solution.
[link]
From: Joel Potischman (Mar 24 2009, at 09:34)
I think it's key to recognize that rebooting, powering down, powering up, etc. are all *processes*, not states. I suppose you could store a VM image of the same OS in each of those states, and posting a representation with state="rebooting" simply swaps out images, but in real-world usage, you are sending requests that the VM needs to execute. Changing the machine's execution state directly, while RESTful, feels wrong because it doesn't map to reality.
I'd implement the resources something like this:
/vms/{vmId}/commands and
/vms/{vmId}/commands/{commandId}
If I want to reboot, I post a Reboot representation to commands. Ditto for Shutdown and Startup commands. My Reboot, Shutdown, and Startup representations might include a message for the system log, an optional delay in seconds before beginning the process, email addresses to send notifications, etc.
So I post a Reboot representation to /vms/33333/commands. I get back 201 Created and a Location header pointing me to /vms/33333/commands/123. I can GET /vms/33333/commands/123 later to check status.
If I want to abort my Reboot, I can attempt to DELETE command 123. If I'm too late, it should return 403 Forbidden. If I try to Reboot or Shutdown an already-turned-off or rebooting VM I should get 409 Conflict.
IMO, this approach is RESTful, idempotent, and most importantly of all, clear in its intent. It doesn't force verb pegs into noun holes, and there is very little impedance mismatch between the model and reality.
[link]
From: John Cowan (Mar 25 2009, at 13:36)
Tim: Like the others said, PUT does have to be idempotent, but you don't want to PUT "reboot", because that is not a state. You want to PUT "down" and then periodically GET the state until you get "down". Then PUT "up" and periodically GET the state "up".
[link]
From: Mike Kelly (Mar 30 2009, at 04:20)
@Paul C. Bryan : Those conditions should be controlled server side, so the action itself is idempotent - that doesn't necessarily mean that it will be successful or valid and this can be indicated by the server response to each request using appropriate HTTP Response codes and headers.
@Joel : There's absolutely no reason that this problem cannot be solved by treating virtual machine power as a state. 'Rebooting' is obviously a state, as are 'on' and 'off' - all of which can and should be treated as idempotent state transitions.
POST is not an idempotent method:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.2
Auditing could easily be implemented by providing some kind of matrix URI for specifying historical 'versions' of the power state.
/power;30/03/09+15.30
[link]
From: Iwamot (Mar 30 2009, at 09:18)
REBOOT isn't a specified method in HTTP. So, IMHO, it's nonsense that you assume sending a "reboot" command isn't idempotent. If the rebooting state is important, you can make a resource which represents it. Or not, you are able to adopt the advice written by Roy.
[link]
From: Joel Potischman (Mar 30 2009, at 11:37)
@Mike, a reboot takes a minute or two because the computer is busy loading programs, etc. I agree that technically "rebooting" is a state, but it's a state that evolves constantly for those few minutes until the state is "on", so I don't think it's particularly meaningful to orient the API that way.
In my opinion, REST APIs that feel more natural are a lot easier to document, use and maintain. I don't doubt that you could solve the problem your way, but I don't think it will feel "right", and I think it will feel even less right as the system evolves.
[link]
From: Mike Kelly (Mar 31 2009, at 12:36)
Treating power as a state is equally flexible as POSTing commands. E.g. Allowing for an additional 'hibernating' state would require trivial changes to logic behind the power URI.
GET'ing/PUT'ing the machine's power state seems a more intuitive exchange, and it's certainly more clear for intermediaries - enabling things like a server side cache invalidation mechanism, which would save processing every GET request on the power (which would be often if it's monitored, presumably) and at the same time ensure that the cache remains completely fresh at all times, since any change of state (i.e. PUT to the power URI) would invalidate the cache immediately.
[link]
From: William Martinez Pomares (Apr 02 2009, at 11:54)
Hello Tim.
You can read my comments at InfoQ
(http://www.infoq.com/news/2009/04/post-state-appropriate#view_40928) and on Roy's blog.
The main question I have if it really you need to map this into REST. My position is that REST is a particular architectural style (not the only one) that may be suitable for some applications and may not be for some other ones. The question is if your APP is.
Tweaking the app semantics to fit one architectural style, implies that style is not for that app. The exercise here is to see if REST actually suites your app.
But, anyway, the Post question may be simpler to answer: one main concern in REST is the remote modification of resources, since you do not know the resource implementation and you actually work with a simple representation, never with the resource. Also, the resource may not be just a file. There is no restriction for a resource to be a process. And Post is meant to allow posting data for a data handling process (see the HTTP spec). What the process does with the data and what changes are performed are of no business to REST.
See? Publish your controller as a process resource, post to it info and be happy. :-)
William Martinez Pomares
[link]
From: Mark Hapner (Apr 08 2009, at 20:07)
Joel Potischman's comment
http://www.tbray.org/ongoing/When/200x/2009/03/20/Rest-Casuistry#c1237912498.216065
captures the core pattern for how hypertext is best used to initiate actions on entities. A cloud cluster is an entity provided by the cloud that 'manages' the lifecycle of the VMs and VNets it contains. This management activity is represented as a list of events. Both the owner and system should post cluster lifecycle events to the tail of the cluster event list. The implementation of the cluster should update these events with status to reflect their execution. The event history is retained to document the cluster lifecycle. The events should likely be cloud lifecycle event microformats. These can be reused in different cloud contexts.
[link]