Join Transform 2021 this July 12-16. Register for the AI event of the year.


Bleacher Report is the third-most visited sports website in the U.S., behind massive brands ESPN and Yahoo Sports (both backed by huge corporations). You don’t get to millions of monthly visitors and peak traffic of 80,000 page requests a minute by ignoring scalability. But the soon-to-be-live 2012 London Olympics promises to triple the independent site’s pageviews.

I talked to Sam Parnell, Bleacher Report’s vice president of engineering, about how he’s preparing for the onslaught.

“We use Ruby on Rails,” Parnell told VentureBeat. “It has a bit of a bad rap as not able to scale, but we’re proof that it can.”

Bleacher Report starts by heavily caching everything that can be cached: images, database content, queries. Caching reduces database fetches and asset retrievals, so the site actually speeds up when the company hits high traffic periods. That’s due to the fact that traffic peaks are almost always associated with single events that are topical and important, reducing the number of assets that need to be cached.

Traffic peaks like that are a good thing for Bleacher Report. Because of its crowdsourced content model, usually the site’s traffic is to very diverse topics.

“What makes us different is how we create content. Most of our competitors have 25-35 journalists and focus on national coverage or popular teams like the New York Yankees,” Parnell said. “We have 2,000 featured contributors, which allows us to generate more sports coverage than anybody else … up to 1,000 stories a day.”

Those thousand stories generate immense traffic, upwards of 90 million monthly visits, and the immense traffic generates massive server load.

To monitor that load, Bleacher Report uses New Relic, an application performance management tool. A somewhat unique aspect of the site’s development model is that engineers continue working on features for their entire lifecycle, from development to production. Every engineer has New Relic open almost full-time, monitoring their parts of the overall site, constantly tuning and optimizing.

“We also spend a lot of time optimizing queries,” Parnell says. “It’s a never-ending task.”

Because Bleacher Report runs on Amazon Web Services, adding additional capacity can be very quick. Parnell can add 50 percent more web serving capacity in 30 minutes.

But just because it’s cloud does not mean that everything is super-simple: Database resources take much more time and configuration to increase. So another thing Parnell does is proactively pre-provision excess database resources.

“Actually, this week we’re scaling up our database layer already,” he told VentureBeat.

The previous record-breaking traffic for Bleacher Report, which peaked at 80,000 requests/minute, was during the Floyd Merryweather-Miguel Cotto boxing match in May. Parnell is expecting three times more for the Olympics. If Bleacher Report makes it, perhaps they’ll start to close with ESPN and Yahoo Sports’ traffic numbers.

Bleacher Report was founded in 2006, and has received about $35 million in funding from Oak Investment Partners, Crosslink Capital, and Hillsven Capital.

Image credit: Pete Saloutos/ShutterStock

VentureBeat

VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:
  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more
Become a member