Bob On Development

According to this interview with a developer at Twitter, that site is currently the largest Ruby on Rails site in operation today. And they are having trouble keeping up with the load. Twitter is even advertising for an engineer with extensive experience in C++.

As Jeff Atwood points out, ROR and its brethren such as Python and PHP have benefited from the increasingly fast hardware out there, but you can still run into a perfect storm like this where you might wish you had a compiled language. Although, I have to be a little skeptical of the benchmarks he cites where ROR would be more than a hundred times slower than typical compiled languages. In a tight and computation-intensive loop, probably so; but as even Atwood admits, this may be as much a database problem as anything.

Twitter’s developer, Alex Payne, mentions that Ruby (or at least, Rails) can’t currently talk to more than one database at a time. That seems like an odd limitation, but he should know if anyone does. So as you add more servers to the web farm, at some point the database requests overwhelm a single database server. Oops.

I’m sure that people will invent faster Ruby interpreters, perhaps based on byte-code compilers, to bring it at least more on a par with the likes of Java or C#. I’m sure that Ruby will develop the ability to talk to multiple databases. It’s just a matter of time.

But this is a cautionary tale — you don’t want to bet the farm on technology that hasn’t been proven to the scale you will likely need to operate at. No matter how cool the code, the customer just wants it to work reliably.

In fairness, I suppose that Twitter has succeeded beyond its wildest dreams, at least in the short run. But right about now I suspect they wish they were running under Java or .NET’s runtime; they’d have a lot more technological elbow room.

Update: David Heinemeier Hansson, aka “DHH”, the primary developer of Ruby on Rails, has posted that Nic Williams has created a 75-line ROR plug-in that solves the multiple database problem. DHH somewhat ungraciously blames Twitter for not taking a look and figuring this out themselves, however, technically at least I have to agree with him. I couldn’t imagine anything inherent in any language or platform that would prohibit access to more than one database. So even though such a simple and obvious thing should have been included in Rails, and it’s telling that until now apparently they’ve felt no need for it, it does seem to be true that Twitter gave up a little too easily. So that’s the other lesson here — don’t assume too quickly that a platform is fundamentally flawed, either, at least to the extent that in an effort to Do Something you try to inject a whole new platform into the mix. That’s expensive and problematic, too.