What
· Technology
· · Cloud
Basic Infrastructure · Recently, I was looking at the infrastructure bills for our CoSocial co-op member-owned Mastodon instance, mostly Digital Ocean and a bit of AWS. They seemed too high for what we’re getting. Which makes me think about the kind of infrastructure that a decentralized social network needs, and how to get it ...
Storage Churn · What are the highest-impact Cloud services? Storage would be near the top of any list. Where by “Storage” I mean what S3 does: Blobs-of-bytes storage that is effectively unlimited in capacity, credibly more durable than anything you could build yourself, and easily connected to the world, either directly or through a CDN. I think we’re entering a period of churn where there’s going to be serious competition on storage price and performance. Which, by the way, is crucially relevant to the Fediverse ...
[4 comments]
[4 comments]
Lock-in and Multi-Cloud · Alone in a room with the customer’s CTO, I said “So, I spent this morning talking to your people, and it looks like you’re using low-level generic tech for everything and managing it yourself. You could save a shit-ton of money by using high-level managed services and get load-scaling for free.”
“Yeah,” he said, “but then we’d be locked in and the next time our enterprise contract comes up for renewal, you can screw us.” ...
[9 comments]
“Yeah,” he said, “but then we’d be locked in and the next time our enterprise contract comes up for renewal, you can screw us.” ...
[9 comments]
Worst Case · Suppose you’re running your organization’s crucial apps in the cloud. Specifically, suppose you’re running them them on AWS, and in particular in the “us-east-1” region? Could us-east-1 go away? What might you do about it? Let’s catastrophize! ...
[20 comments]
[20 comments]
Workflows in AWS and GCP · Recently, Google launched a beta of Google Cloud Workflows. This grabs my attention because I did a lot of work on AWS Step Functions, also a workflow service. The differences between them are super interesting, if you’re among the handful of humans who care about workflows in the cloud. For those among the other 7.8 billion, move right along, nothing to see here ...
[6 comments]
[6 comments]
Content-based Filtering · The publish/subscribe pattern is central to data in motion — event-driven and messaging-based apps, I mean. I’m increasingly convinced that pub/sub software just isn’t complete without some sort of declarative filtering technology, so that you can subscribe to a huge shared torrent of data and only see the parts of it that you need to process. You could look at everything and write code to reject the data you don’t care about, but it’s nice to write a declarative rule and have the system take care of the filtering for you ...
The Cloud and Open Source · This is eventually about the public cloud and Open Source, but — apologies in advance — takes an indirect and long-ish path ...
[6 comments]
[6 comments]
Google Photos Breakage · The camera systems in the Google Pixel phones are excellent, but Google makes it way too hard for the serious photographer to get a decent workflow going. Does someone out there know a better way? If not, let’s get together and yell at Google. [Update: I found a good way to do this.] ...
[11 comments]
[11 comments]
EventBridge · The launch of Amazon EventBridge, a somewhat but not entirely new thing, has been well-covered by Jeff Barr; if you want to know what it is, go read Jeff. This piece is to provide a bit of background and context on EventBridge. I didn’t actually make any direct contributions, but was upstream from this work at the definition and early-planning stage ...
On SQS · In my position I probably shouldn’t have a favorite AWS product, just like you shouldn’t have a favorite child. I do have a fave service but fortunately I’m not an (even partial) parent; so let’s hope that’s OK. I’m talking about Amazon Simple Queue Service, which nobody ever calls by its full name ...
[3 comments]
[3 comments]
Serverlessness · I work in AWS’s Serverless group, and in the process of pulling together my presentation at re:Invent, discovered that I have a lot of opinions on the subject and, while they may well be wrong, are at least well-informed. You can watch that YouTube, but who’s got an hour to spare? And anyhow, blogging’s really my favorite medium, so here we go. If I tried to glom them all together into one mega-essay it’d be brutally long, so let’s go short-form ...
[3 comments]
[3 comments]
SF-5: Serverless Bills? · One of the best reasons to go serverless is that you might save a lot of money. This is especially true when your load is peaky and has quiet times; because when your infrastructure isn’t working, you’re not paying. But, not all loads are peaky. And here’s a quote from an AWS internal mailing list: “For every compute load, there’s some level of TPS where Lambda is going to be more expensive than servers.” So, when is that? And how much should you care? ...
SF-4: Serverless Latency? · Suppose we like the idea of going serverless (we do). What should we worry about when we make that bet? What I hear when I talk to people thinking about it, mostly, is latency. Can a run-on-demand function respond as quickly as a warmed-up Web server sitting there in memory waiting for incoming work? The answer, unsurprisingly, is “it depends” ...
[5 comments]
[5 comments]
SF-3: Serverless Everything? · Sometimes we fans get a little over-excited and declaim that everything should be serverless. After all, we’re pretty convinced that owning data centers and computers is becoming a thing of the past. Well then, how could configuring your own hosts and paying for them even when they’re not working ever be a good idea? Let’s try to be moderate and pragmatic: Serverless, where possible ...
[1 comment]
[1 comment]
SF-2: Why Serverless? · Well obviously: Frugality, security, and elasticity. But I want more, I want better software ...
[1 comment]
[1 comment]
SF-1: What Is Serverless? · I don’t think it’s that complicated: If you can’t see the servers in the service, then it’s serverless. Yeah, they’re still there, but the whole point is that you can mostly not worry about them ...
Step Functions Integration · On Thursday we launched some add-ons for AWS Step Functions, on which I helped a bit. As usual, there’s a nice Jeff Barr blog. This is to add design notes and extra color ...
[1 comment]
[1 comment]
Why Serverless? · We were arguing at work about different modes of computing, and it dawned on me that the big arguments for going serverless are business arguments, not really technology-centric at all. Maybe everyone else already noticed ...
[3 comments]
[3 comments]
IsItOnAWS Lessons · I did some recreational programming over Christmas and the blog I wrote about it is now guesting in Jeff Barr’s space for your amusement; try the software at IsItOnAWS.com. What I didn’t do there was relay the lessons I picked up along the way; one or two are around AWS, but most follow from this being my first nontrivial expedition into the land of NodeJS. So (acknowledging that only 0.8% of my profession aren’t already Nodesters), here they are. Spoiler: I don’t like Node very much ...
[5 comments]
[5 comments]
Two AWS Years · Wow, it was December 2014 when I climbed on board this train. I’m sitting in a pretty interesting place and feel I owe the world some reportage ...
[5 comments]
[5 comments]
Network Storage · A couple days ago in New Home Network I posted a request for advice on a home NAS box and networking hardware. Now I have the storage box, and boy was it ever easy and straightforward and anxiety-relieving. If you haven’t done this already, you might want to ...
[11 comments]
[11 comments]
Serverlessness · Today Microsoft announced Azure Functions, joining Google Cloud Functions and (from 2014) AWS Lambda. This is fun stuff, and might be a big deal ...
[3 comments]
[3 comments]
Cloud Eventing · So, I helped build Amazon CloudWatch Events (blog, AWS console), which just launched. Been a while since my last extended spell of being an actual software engineer. Shipping feels good ...
[1 comment]
[1 comment]
One Amazon Year · December first made it a year here at Amazon Vancouver’s engineering castle in the sky. I’m working with good people in a cool office on interesting stuff. It’s at the white-hot center of server-side computing but surprisingly unsurprising ...
[10 comments]
[10 comments]
re:Invent in Vegas · That was intense. AWS’s customers are great. But I wish I could like Vegas more ...
Amazonian · As of December I’m working for Amazon. This will be different from my last few gigs; in particular, don’t expect to read about it here ...
[20 comments]
[20 comments]
Slow REST · We’re working on a fairly substantial revision of the Sun Cloud API, motivated by this problem: In a RESTful context, how do you handle state-changing operations (POST, PUT, DELETE) which have substantial and unpredictable latency? ...
[26 comments]
[26 comments]
