Bleacher Report is the third-most visited sports website in the U.S., behind massive brands ESPN and Yahoo Sports (both backed by huge corporations). You don’t get to millions of monthly visitors and peak traffic of 80,000 page requests a minute by ignoring scalability. But the soon-to-be-live 2012 London Olympics promises to triple the independent site’s pageviews.
I talked to Sam Parnell, Bleacher Report’s vice president of engineering, about how he’s preparing for the onslaught.
“We use Ruby on Rails,” Parnell told VentureBeat. “It has a bit of a bad rap as not able to scale, but we’re proof that it can.”
Bleacher Report starts by heavily caching everything that can be cached: images, database content, queries. Caching reduces database fetches and asset retrievals, so the site actually speeds up when the company hits high traffic periods. That’s due to the fact that traffic peaks are almost always associated with single events that are topical and important, reducing the number of assets that need to be cached.
Traffic peaks like that are a good thing for Bleacher Report. Because of its crowdsourced content model, usually the site’s traffic is to very diverse topics.
“What makes us different is how we create content. Most of our competitors have 25-35 journalists and focus on national coverage or popular teams like the New York Yankees,” Parnell said. “We have 2,000 featured contributors, which allows us to generate more sports coverage than anybody else … up to 1,000 stories a day.”
Those thousand stories generate immense traffic, upwards of 90 million monthly visits, and the immense traffic generates massive server load.
To monitor that load, Bleacher Report uses New Relic, an application performance management tool. A somewhat unique aspect of the site’s development model is that engineers continue working on features for their entire lifecycle, from development to production. Every engineer has New Relic open almost full-time, monitoring their parts of the overall site, constantly tuning and optimizing.
“We also spend a lot of time optimizing queries,” Parnell says. “It’s a never-ending task.”
Because Bleacher Report runs on Amazon Web Services, adding additional capacity can be very quick. Parnell can add 50 percent more web serving capacity in 30 minutes.
But just because it’s cloud does not mean that everything is super-simple: Database resources take much more time and configuration to increase. So another thing Parnell does is proactively pre-provision excess database resources.
“Actually, this week we’re scaling up our database layer already,” he told VentureBeat.
The previous record-breaking traffic for Bleacher Report, which peaked at 80,000 requests/minute, was during the Floyd Merryweather-Miguel Cotto boxing match in May. Parnell is expecting three times more for the Olympics. If Bleacher Report makes it, perhaps they’ll start to close with ESPN and Yahoo Sports’ traffic numbers.
Bleacher Report was founded in 2006, and has received about $35 million in funding from Oak Investment Partners, Crosslink Capital, and Hillsven Capital.
Image credit: Pete Saloutos/ShutterStock