May 23, 2008
140 characters should be enough for everybody
Rails would do itself no harm by conceding that it isn’t a platform that can compete with Java or C when it comes to intensive tasks.Oh boy.
This quote comes from a recent article on Techcrunch, and although it wasn't penned by Michael Arrington, it's guaranteed to generate quite a discussion.
I find the post reasonably fair toward Ruby on Rails (even though the words "Ruby on Rails can't compete with Java or C when it comes to intensive tasks" are probably going to get the Rails' community collective blood boiling): credit is given to Rails for being an enabling technology, but the conclusion that I summarized above is certainly going to score high in Google searches in the coming months for anyone interested in finding out about Rails' scalability. It is a big deal.
It's difficult to point the blame about what's going at Twitter, especially since the Twitter engineers haven't been very transparent about what's going on (they seem determined to correct this), but I can make a couple of observations:
- It's probably a bit too early to blame Ruby on Rails for the horrible uptime that Twitter has shown so far, but since Twitter is moving away from Rails, we'll probably never know. And the worst part is that Rails will never get a fair chance to be exonerated.
- The excuse that this kind of scaling problem is unprecedented makes no sense. There are plenty of successful IM platforms in production today that are tackling, and successfully solving, the exact same problems that are stumping Twitter. Of course, they also have massive resources (hardware, software and human), which makes the comparison a bit unfair, but the technical assessment still stands: there are known and documented solutions to these problems, so it's purely a matter of Twitter giving themselves the means to implement them.
First, the community seems to be very eager to help. Unfortunately, these attempts are misguided or misinformed more often than not (possibly including this very post), but here is what I think is the most insightful article on this problem space so far.
Second, the Twitter people are not only becoming more transparent in their communications, they also seem to follow the public discussions quite closely. Hopefully, they will be able to figure out the next steps and, more importantly, they'll receive the means to implement them (how about this for a start? :-)).
There's certainly been an enormous amount of vitriol and cluelessness on both sides of this 'debate'. Sadly most of the commenters tend to be ambulance chasers promising that their framework will magically solve all twitter's problems with a few hours work. While I'm sure there are people within the rails community which have claimed that rails and ruby would be a good fit for twitter's backend message routing code, I'm not one of them. It's all about the right tool for the right job.
The problems of routing and delivering high volumes of messages isn't exactly a problem that we try to solve with Rails, there are other tools and languages more suited to this. Having twitter rewrite a portion of their backend code in some other language is no more a problem for rails than having most java apps persisted in relational databases is a problem for JavaEE.
Finally, ev has already corrected that techcrunch article about them 'moving away from rails'. They already have portions of their message handling code written in something else, and now they're changing it further. As far as anyone's aware they intend to continue using rails as their frontend, at least for now.
-- Koz - rails guy
Posted by: Koz at May 23, 2008 10:16 PM> There are plenty of successful IM platforms in production
> today that are tackling, and successfully solving, the exact
> same problems that are stumping Twitter.
Existing IM services do not have anything like the per user home pages in Twitter.
Posted by: Jabber Dabber at May 23, 2008 11:18 PMA second, Twitter guys have NOT said that they move away. It is very unfair to claim they are lying.
Either you trust them, or you need to accuse them of lying. But this dagger and dirt throwing is annoying.
Posted by: markus at May 24, 2008 04:33 AMGiven the constant Twitter downtime and their inability to fix scalability issues for whatever reasons, its only a matter of time a "better" Twitter comes and takes over the existing twitter users.
There are some pretty indecisive people sitting on Twitter management. Twitter management, make a decision will ya?
Posted by: Muthu Ramadoss at May 25, 2008 07:42 AMfrom this post dev.twitter.com/2008/05/twittering-about-architecture.html
it's clear that twitter tries to move away from the db backend (since their app is not a CMS as they realized now).
what's the point of using rails (or any web frameworks) without a database backend?
personally i would be surprised if they used a generic web framework at all in the new version.
Posted by: poko at May 25, 2008 08:56 AM
Hi Cedric,
You are missing the point. Twitter has benefited HUGELY from Rails already. The reason: TIME TO MARKET!!!
Whilst other whre still chrunching out code, Twitter was out there gaining brand recognition and market share. You see the consumer can only deal with one or two major brands in each market segment. It's called brand recognition. So theres Amazon for Books, Face Book for social networking and now Twitter for instant messaging based networking.
They have followed the Agile mantra of deliver early and frequently. Even if they choose tto refactor all the Ruby code to C over time :) then they would still be quids in.
We need to be more responsive to business needs. Time to market and the response time to changes in the market is a key factor. Rails delivers on both of these.
Lets assume that Ruby doesnt scale (although performance issues tend to be due to poor design, so I'm not convinced that Ruby or Rails are to blame), for lots of scenarios Rails is more than good enough to gain that initial market presence.
Having more customers than you can deal with is the type of problem most businesses dream of. LOL!!!
Paul.
Posted by: Paul Beckford at June 2, 2008 10:02 AMI still think the problem is because they used Rails, but in a somewhat indirect way.
To me a web developer using Rails is like a cook baking cakes using a Duncan Hines or Betty Crocker cake mix in a box. Sure, you can bake that cake in no time at all (great "time to market"), but you're learning very little about baking.
Posted by: lumpynose at June 4, 2008 06:04 PM