Micro-messaging site Twitter has gone from zero to transparent in a hurry. That is to say, it has gone from keeping users in the dark during its downtime to explaining the problems it is having in detail. Tonight, developer Alex Payne wrote a Q&A post on the Twitter Developer Blog in which he answered some users’ questions about Twitter’s recent woes.
One particularly interesting question was the following:
charles asks if there’s anything users can do to lighten our load.
To which Payne responded:
The events that hit our system the hardest are generally when “popular” users - that is, users with large numbers of followers and people they’re following - perform a number of actions in rapid succession. This usually results in a number of big queries that pile up in our database(s). Not running scripts to follow thousands of users at a time would be a help, but that’s behavior we have to limit on our side.
Most users of Twitter will single out one person who this points to: blogger and Fast Company employee Robert Scoble.
Scoble, with his 25,000+ followers and 21,000+ people he is following is a beast on the service. I would consider myself a fairly heavy user of the service and I’ve sent 3,598 tweets (Twitter messages) — Scoble has sent 12,318. This is clearly putting a strain on the service.
Perhaps more interesting was that in multiple responses, Payne more or less said not to blame Ruby on Rails (RoR) for Twitter’s woes. A lot of commenters on this blog and others are quick to jump on and blame RoR, which has even led to rumors that Twitter was abandoning it. These turned out not to be true, and in fact Twitter will continue to use RoR for the time being — at least for its front end work.
Payne also said he would use Rails again if he were looking for rapid prototyping, as Twitter was. (Payne was not around however during the initial building of Twitter.) That idea is in line with a post Nati Shalom, GigaSpaces’ chief technology officer, wrote earlier this month detailing his thoughts on Twitter’s problems from a back-end perspective.
I had a chance to talk with Shalom today about his post and his thoughts on Twitter’s woes. He believes the problem lies in the explicit contradiction between scaling and time to market. It’s also a problem with not knowing if a company will be successful and as such being cautious with spending money and time upfront to build something slowly that is fully scalable.
