When times are tough, money is tight. Which means, you’d think, that the golden era of Cloud Computing, as in pay-as-you-go infrastructure, is upon us. It should be, but we’re not there yet. [This is part of the Tough Times series.]
As I noted last time, the current economic climate is going to get in the way of anything that requires laying out capital. In this light, cloud computing starts to look good for the same reason that Open Source looks good: low up-front costs. So, just like everyone else, I think technology providers and consumers need to be looking really hard in this direction.
Tech Issue · But there are two problems. The small problem is that we haven’t quite figured out the architectural sweet spot for cloud platforms. Is it Amazon’s EC2/S3 “Naked virtual whitebox” model? Is it a Platform-as-a-service flavor like Google App Engine? We just don’t know yet; stay tuned.
Big Issue · I mean a really big issue: if cloud computing is going to take off, it absolutely, totally, must be lockin-free. What that means if that I’m deploying my app on Vendor X’s platform, there have to be other vendors Y and Z such that I can pull my app and its data off X and it’ll all run with minimal tweaks on either Y or Z.
At the moment, I don’t think either the Amazon or Google offerings qualify.
Are we so deranged here in the twenty-first century that we’re going to re-enact, wide-eyed, the twin tragedies of the great desktop-suite lock-in and the great proprietary-SQL lock-in? You know, the ones where you give a platform vendor control over your IT budget? Gimme a break.
I’m simply not interested in any cloud offering at any level unless it offers zero barrier-to-exit.
PHP ·
You know, there is already sort of a cloud ecosystem out there in the world
of PHP. There are a whole bunch of competitive vendors where you can upload a
bunch of .php files and database dumps and with only a
moderate amount of twiddling, get your app running.
I don’t think that PHP is the best way to build Web apps. But increasingly, I’m developing an appreciation for its ecosystem.
Contributions
Comment feed for ongoing:
From: David Petherick (Oct 14 2008, at 16:50)
Tim, I agree that it's incredible that the vendors can't see that a low barrier to exit is critical - nobody who is concerned for the future welfare of their organisation's IT competitiveness (and that's CEO, CTO, CIO and others) will want a 'lock-in' or an extraction cost.
To give an example of how easy PHP is, I moved a whole stack of PHP running several sites a month ago - nothing had to change when it moved from London to California - not one line of code. And the transfer was provided by the web hosts, who had scripts to run the whole thing. Took me 7 minutes to move 30+ web sites and back ends.
By the way, in case you didn't get to listen yet, here's our interview at Future of Web Apps in London after your keynote: http://thenextweb.org/2008/10/10/top-tips-for-tough-times-from-tim-bray-at-fowa/
Regards from Scotland, David
[link]
From: Doug Cutting (Oct 14 2008, at 16:54)
I hear that Amazon's stuff is getting a *lot* more traction than Google's. As for lock-in, APIs might not be the gating factor but rather data locality. When your data is in Amazon's cloud, it's faster and cheaper to pass it to other services in that same cloud. That's a hard economy to escape. Amazon's margins are also low enough that folks will have trouble competing. So I can see a new sort of lock-in: all of your code could be portable to different cloud platforms, but there's only one cloud that can run them efficiently and economically.
[link]
From: Chris Mahan (Oct 14 2008, at 17:01)
So essentially what's missing are "whole website" conversion scripts.
And yeah, the PHP + MySQL model is fairly compelling.
[link]
From: John Cowan (Oct 14 2008, at 17:03)
I don't think there's any lock-in around Amazon EC2 except for whatever connections to Amazon S3 your code has -- which would be easy to change to some other storage back end in any well-written programs. Other than that, you could pull off your EC2 code and run it in any sensible datacenter, physical or virtual. If EC2 is single-source today, it's because nobody else happens to have the combination of resources and chutzpah needed to compete with it eyeball to eyeball.
I know someone who used to run his business (customized Web crawls) off his home computer; when he got too many customers for that, he stepped up to EC2 with nary a change in code required except to add the aforementioned S3 storage layer.
[link]
From: Jon Ellis (Oct 14 2008, at 17:12)
If capital outlay is a problem for the user of the cloud, why do you think it won't be for the cloud providers?
After all, they have to make a speculative investment to build the thing...
[link]
From: Michael Davidson (Oct 14 2008, at 17:28)
There are already efforts underway to make drop-in replacements for AppEngine. See, e.g., http://appdrop.com/.
I am hopeful that more, better replacements will pop up as time goes on.
[link]
From: Justin Watt (Oct 14 2008, at 17:30)
"There are a whole bunch of competitive vendors..."
You mean like every webhost on the planet? Or something else?
[link]
From: DeWitt Clinton (Oct 14 2008, at 17:39)
Hi Tim,
You've mentioned the lock-in question before, and I believe we're in agreement on the principle.
Could you articulate specifically what it would mean for these services to be lock-in free? What is currently missing in the service agreements and licenses, and what specifically would need to be changed to pass that bar? I.e., what do you mean by a "zero barrier-to-exit"? That would help companies like Google and Amazon design their services with this goal in mind.
Thanks, Tim!
-DeWitt
[link]
From: zack (Oct 14 2008, at 18:20)
I'm not sure why Amazon wouldn't qualify regarding Issue #2. Care to clarify that bit? They are basically giving you hardware and saying, figure out how to make it do what you want. No?
[link]
From: Gavin Terrill (Oct 14 2008, at 19:00)
The last time this happened I remember we got a little creative on how our big company customers could buy our software. Essentially, we did a deal with a financing company that allowed our customers to lease the product. Our customers liked it as it meant monthly payments, while we still got our money up front. At the end of the lease, the payout figure was negligible. There were various penalties for an early exit, but luckily none of our customers took the option.
On the cloudy front, another option would be the "roll your own app" offering ala Ning. Let a hundred thousand social networks bloom! A Notes equivalent (like http://creator.zoho.com/ for example) in the cloud is the next logical step. Wholeheartedly agree that it would be a non-starter if it means lock-in though.
[link]
From: Another Tim (Oct 14 2008, at 20:56)
Yet another issue: There are laws how und where to process and store privacy-related (= almost all) data. Amazon tries to adhere by S3-EU, but Google? Nada. The Cloud may be high above us, yet it is not a rightless and lawless enviroment.
[link]
From: Tony Fisk (Oct 14 2008, at 21:30)
<i>Are we so deranged here in the twenty-first century that we’re going to re-enact, wide-eyed, the twin tragedies of the great desktop-suite lock-in and the great proprietary-SQL lock-in? You know, the ones where you give a platform vendor control over your IT budget?</i>
I certainly do know! I have spent much of the last 3-4 years developing in-house software to work around and enhance the ossified limitations of a locked-in black box purchasing suite, with non-existent support.
...using PHP!
(and AJAX, and MySQL, and....)
[link]
From: Tim Bray (Oct 14 2008, at 23:46)
DeWitt: I thought I was pretty specific. If there's somewhere I can copy my code & data to and and it'll work (maybe not Just Work, but with a modest amount of effort), then I'm not locked-in.
[link]
From: DeWitt Clinton (Oct 15 2008, at 08:21)
Tim,
That's why I asked -- you can already do that in both the case of Amazon's services and App Engine. Sure, in the case of EC2 and S3 you'll need to find a new place to host the image and a new backend for the data, but Amazon isn't trying to stop you from doing that. (Actually not sure about the AMI format licensing, but I assumed it was supposed to be open.)
In App Engine's case people can run the open source userland stack (which exposes the API you code to) on other providers any time the want, and there are plenty of open source bigtable implementations to chose from. Granted, bulk export of data is still a bit of a manual process, but it is doable even today and we're working to make it even easier.
Ae you saying that lock-in is avoided only once the alternative hosts exist?
But how does Amazon or Google facilitate that, beyond getting licensing correct and open sourcing as much code as they can? Obviously we can't be the ones setting up the alternative instances. (Though we can cheer for them, like we did when we saw the App Engine API implemented on top of EC2 and S3.)
To Doug Cutting's very good point, the way Amazon and Google (and everyone else in this space) seem to be trying to compete is by offering the best value, in terms of reliability (uptime, replication) and performance (data locality, latency, etc) and monitoring and management tools. Which is as it should be.
Please feel free to shoot me a note to discuss further if you'd like.
-DeWitt
[link]
From: Mark (Oct 15 2008, at 08:51)
Unsurprisingly, I agree with you about the lock-in issue. But the really scary issue is not lock-in, but lock-out. As in, what recourse do you have if your service provider suddenly decides that your account is exhibiting suspicious activity, and they disable your account? Go to your favorite search engine and look for "Google account disabled" or "locked out of Google account". (Or, if you like, change "Google" to your favorite service provider.)
I think that most people implicitly assume that they have some inherent right to continue using a service provider, as long as the company is still in business and the account is actively in use (especially if you're paying for the service, but even if you're not). These people are generally shocked to learn that their access can be revoked for no outwardly apparent reason, that they may never learn the real reason, and that they have little recourse but to start over. With backups, assuming they made any.
Certainly, contracts can be written to protect against this. But the default terms of service are astronomically weighted in favor of service providers, and the process for recovering your account ranges from byzantine to Kafka-esque.
[link]
From: Ciaran (Oct 15 2008, at 09:53)
"I’m simply not interested in any cloud offering at any level unless it offers zero barrier-to-exit."
Ah, you say that, but I see that one of the following accounts is active, and the other is not so much. ;)
Locked up: http://twitter.com/timbray/
Open: http://identi.ca/tbray
[link]
From: Barton George (Oct 15 2008, at 11:44)
Hey Tim,
In my new gig Im working to promote a cloud-based process documentation solution named "Blueprint." A while back the team decided to with Rackspace for Blueprint hosting solution, one of the main reasons being that they have SAS70 certification. I'm curious as to why Rackspace are not often mentioned in cloud computing discussions along amazon and google?
Now that you've brought it up, I'm interested in checking out what Barriers to exit rackspace may or may not have.
mahalo
[link]
From: John Cowan (Oct 15 2008, at 11:48)
I admit I don't know for sure if Amazon has covert anti-abuse provisions (I assume Google has -- I work for Google but don't know much about AppEngine), but the most obvious anti-abuse provision for EC2 and S3 is that you pay for the bandwidth, storage, and cycles you use. You abuse it, youpay the bill.
There's something very appealing about that business model, the analogue of Fat Pipe, Always On, Get Out Of The Way. Granted, people tend to prefer flat rates to metered ones, but the meter ticks very slowly (US$0.10-0.20 per hour per basic instance, gigabyte transmitted, or gigabyte per month stored).
[link]
From: Bill de hÓra (Oct 15 2008, at 15:07)
"Are we so deranged here in the twenty-first century that we’re going to re-enact, wide-eyed, the twin tragedies of the great desktop-suite lock-in and the great proprietary-SQL lock-in?"
Great soundbite, but...
...there's going to be too much data to unlock. It'll be economically (and ecologically) cheaper to move processing to data to reduce transfer costs.
If data locality wins, then this means raw storage providers win as long as they can host compute close to the data making the problem a function of data transfer. Remember what your Sun colleague said - storage is out of control. I think this is why S3 is beating out GAE; it has a much better story for compute/data/transfer tradeoffs.
This is what our industry should be worried about today, not the current debt paralysis and hard times. Data is our number one challenge and responsibility.
[link]
From: daaku (Oct 15 2008, at 23:03)
Isn't the cloud already here? Depending on how big you are, you 1) make your own cloud 2) rend space in someone's cloud 3) rent hardware in someone's cloud 4) rent virtualized hardware in someone's cloud. Where cloud = datacenter.
[link]
From: Rob Sayre (Oct 17 2008, at 17:49)
I don't buy it.
I think Amazon and Google infrastructure might be a great way to go if you must serve all of your data over the Web. Otherwise, why bother? SmugMug is a good example that works with this strategy.
If you're forced to store data, it seems like a good bet to assume that data is worth more than datacenter that stores it. There will be outliers, like SmugMug, but most valuable data carries a risk of disclosure. I don't think you will see your bank account outsourced to S3 any time soon.
[link]
From: Geir Magnusson Jr (Oct 20 2008, at 04:15)
I don't think that the Google lock-in problem is solved for a few reasons. (I think about it quite a bit as we're trying to solve the problem at 10gen, where I work).
First, are any of the open source implementations of BigTable compatible enough and reliable enough? I'm sure they will be in time, and I'm not holding Google responsible for the quality and reliability of their competitors, but until they are, there's a de-facto lock-in for anyone who is serious about their data : putting the issue of data migration aside, if there's no trustable implementation of the data store API outside of Google's, you're stuck.
Second, AE developers are offered quite a few proprietary APIs that are fundamental to app development - e.g. the User API - that are connected to internal Google infrastructure. It's true that these APIs are optional and a dev can choose to use them or not, but if you do, you're... stuck.
I think the opportunity here is to create a "standard", minimal set of APIs and services that are supported by "cloud" application platforms to help ensure portability. We've been trying to get this rolling on the server-side JavaScript side of things, and it makes perfect sense to try to generalize to other language platforms and frameworks.
If someone is interested, ping me :)
geir
(geir ascii(0x40) 10gen ascii(0x2E) com)
[link]
From: Jason (Oct 21 2008, at 06:34)
I think the whole lock-in factor has two faces: Platform flexibility with regards to interoperability and secondly the data and reliability/service guarantee of the cloud-data center itself.
I believe GoogleApp Engine does not have any intention to store a user application data in their cloud and lock it. It is part of a larger plan. These platforms are built factoring specific cloud environments where it will reside and run. Soon they should be able to jump between clouds and give more options to the customer.
One of my colleague tried out Wolf Frameworks app_development_Platform (www.wolfframeworks.com) and these guys are even giving the flexibility of having the data posted in our own database. They have enbaled the beta release on IE only, but hey -- the application is sitting in a cloud, designer in the cloud, UI is browser generated (AJAX / Javascript) & data in our secured database and have exposed business rules to manage all of them together including external web services calls, etc. Works good for now, takes care of my lockin stuff.
[link]
From: Paul Mison (Oct 26 2008, at 16:12)
I've got a fair bit of experience in PAAS apps, albeit from a personal not commercial perspective. I've written apps for Zimki, a now-defunct JavaScript platform, and for Google App Engine. This week I ported the code I wrote for Zimki to AppJet, which also uses JavaScript but is, naturally, somewhat different.
It took probably a solid day of coding to port my (fairly trivial) app from one to the other. It helped that I wasn't really using storage much, but if I had been, it probably wouldn't have been that much harder- most of the PAAS storage models are very much row-based and non-relational. It helped a great deal that both platforms used the same language and that I was able to use the same template implementation on both; without that, I suspect it would have taken three or four times as long, which is verging on "reimplement from scratch" level.
Short summary, then: there's no easy migration between PAAS providers, but it is possible, at least in some cases.
[link]