Facebook explains secrets of building hugely scalable sites

Image Credit: Shutterstock

Facebook, which joined up with Ericsson, MediaTek, Nokia, Opera, Qualcomm, and Samsung to launch Internet.org a few weeks ago, has already produced something for the ostensibly public-minded initiative.

It has published a white paper detailing some of its secrets to success.

Internet.org launched on August 20 as a site dedicated to helping the two-thirds of the world that currently has no Internet access — approximately 5 billion people — get connected.

Of course, getting those people on board the sharing and liking train would benefit Facebook, which has “only” 1.1 billion customers so far. It would also be great for the other companies in the partnership, who stand to benefit by selling the hardware and software those billions use to get online. Perhaps that is why Internet.org is not described, anywhere in its literature, as a nonprofit: It appears to be a promotional site, pure and simple, for the expansion of these companies’ markets.

That said, Facebook’s new, 70-page white paper contains a wealth of detail about how Facebook has already solved numerous scaling challenges — detail that will be useful to anyone who is trying to build a business capable of serving billions.

The company boasts that, every day, its users share 4.75 billion content items on Facebook (including status updates, wall posts, photos, videos and comments), make more than 4.5 billion “Likes,” and send more than 10 billion messages. What’s more, the company says, users have uploaded 250 billion photos to Facebook, a number that grows by more than 350 million photos every day, on average.

In all, Facebook stores 250 petabytes of data in its data centers, and more than half a petabyte of new data arrives each day.

An illustration from Facebook's white paper showing how it tracks data center efficiency.

Above: An illustration from Facebook’s white paper showing how it tracks data center efficiency.

Image Credit: Facebook

And it does it all with some of the most power-efficient data centers on the planet.

Want to be like Facebook? Read this white paper. It delves quite deeply into some of the technical challenges the company faced, and how it solved them.

For example, Facebook describes how it built a tool called HipHop (now an open-source project on GitHub) to transcode easy-to-write PHP code into faster-to-execute C++ code. It later built a virtual machine to execute PHP programs faster — 5 times faster than Zend PHP, according to Facebook.

Other open-source technologies that Facebook developed or contributed to include Corona, a tool for more efficiently scheduling computational jobs; AvatarNode, a tool for managing and coordinating write operations to the HDFS file store; and a customized version of Apache Giraphe.

(Facebook’s open-source hardware framework is the basis of its data center in Prineville, Ore., and we’ve been covering that since its earliest days.)

In the white paper, Facebook also describes the ways it is optimizing its mobile apps for use on a wide variety of devices, with many different capabilities, in different languages and different parts of the world.

The company has built a tool it calls Air Traffic Control to simulate different network and device conditions right from its headquarters in Menlo Park, Calif. Using the results of these tests, it has been able to optimize things such as image sizes to deliver the most efficient possible package to each user, without overloading their network connections.

Facebook also describes how it acquired Snaptu in 2011 to provide a version of Facebook optimized for feature phones. “Facebook for Every Phone” is now used by 100 million people worldwide, the company says.

In the later sections of the white paper, Facebook describes efforts by Qualcomm and Ericsson to make the network technologies underlying the wireless Internet even more efficient. Qualcomm, for its part, puts that in the context of a “1000x Challenge” — the challenge to build “a world in which there is sufficient global wireless capacity to meet demand that is 1,000 times greater than today.”

If you want to know how to build a business that can reach billions of people, many of whom don’t have smartphones, and which can deal with the massive influx of data that a customer base like that will generate, this white paper is a good start.

More information:

Facebook is the world’s largest social network, with over 1.15 billion monthly active users. Facebook was founded by Mark Zuckerberg in February 2004, initially as an exclusive network for Harvard students. It was a huge hit: in 2 w... read more »

Powered by VBProfiles


VentureBeat is studying the state of marketing technology. Chime in, and we’ll share the data.
0 comments