Systems Programming in the Cloud

Tim Bray has an article up about static versus dynamic languages, and why he finds the static ones less annoying for Android programming than for web programming. It’s a good article.

Something I’ve noticed over the past few years of web programming is how it’s slowly becoming more like the systems/realtime/embedded programming I once did. Not exactly like it, but similar.

In the early days the difference was stark: as Tim points out, the stereotypical web app is an input/output machine. Requests go in, responses go out. Everything you do in a web app occurs somewhere in between receiving a lump of data and pushing out another lump of data. You don’t have to coordinate multiple threads; the app server does that for you. You don’t have to worry about race conditions; database transactions take care of that for you. It’s a black box: data goes in one end, different data goes out the other.

The big difference Tim sees between web programming and device programming is the number of APIs you typically have to deal with:

  1. Touch-screen in­ter­ac­tions.
  2. Tele­phony.
  3. More radio in­ter­faces, po­ten­tially: WiFi, NFC, and Blue­Tooth.
  4. A GPS and com­pass and maybe al­time­ter.
  5. Audio gear, in­clud­ing speak­ers and a mi­cro­phone.
  6. A cam­era, with a sen­sor and lots of con­trols.
  7. An ac­celerom­e­ter.
  8. Last but not least, a vi­bra­tor.

The thing is, web programming is starting to look more and more like this, as web programs become nodes in ever more complex constellations of services.

  1. Semi-smart clients which are asynchronously pulling data, or expecting you to push data, via Websockets and other technologies.
  2. Multiple data stores (e.g. Postgresql and Redis) which may become inconsistent if you’re not careful.
  3. A service for sending and receiving email.
  4. A service for sending and receiving SMS messages, IMs, or voice calls.
  5. Integration with a half-dozen social networks
  6. Dedicated work queues for long-running operations

And much like the many data busses embedded programs have to coordinate (RS-232! USB! Ethernet! 1553!), often each of these external  services requires a slightly different kind of impedance-matching code in order to talk to it.

What’s especially reminiscent of  systems programming is that any one of these services can fail. In the primordial web app, a failure of the hardware hosting the app, or the database backing the app, is Not Your Problem. The request either works; or it has a software bug (which is your problem); or the platform fails (which is some poor overworked sysadmin’s problem). But when your app depends on a half a dozen services, any of which could crash, or get really slow, or go just plain wonky at any time, you have to start thinking about failure modes and exception safety and sanity checking and periodic self-tests.

I’m not sure if this new “cloud systems programming” will prompt a resurgence of interests in static languages. But I think it is causing a lot of programmers to rethink how web programs are designed. And a lot of the new thought is eerily familiar. For instance, a lot of web programmers appear to be rediscovering that RPC is a terribly leaky abstraction whether done via UNIX domain sockets or HTTP, and why systems that exchange fire-and-forget messages are almost always preferable.

On the whole, I’m excited about the challenges of heavily interconnected web applications. Yes, it brings with it a lot of the headaches I thought I’d left behind in the systems programming world. But these are interesting problems to solve. And there are few better feelings as a programmer than looking at a system happily humming away, knowing that inside there is an intricate dance going, dozens of inputs being handled and coordinated and dispatched, all according to your careful choreography.

1 comment

  1. I’ve been having similar thoughts, and moreover have been amazed at how many lessons from the systems programming world have been historically ignored by web developers and are now coming home to roost.  You mentioned avoiding race conditions by wrapping database interactions in transactions.  The number of systems that I’ve seen that don’t do that, including ORMs, however, or even do something simpler, like use a unique constraint when available in the underlying RDBMS, is quite shocking.

Leave a Reply

Your email address will not be published. Required fields are marked *