Hadoop in the wild: These are the 3 biggest challenges

When we deployed the first production Hadoop cluster in 2006, we were looking to build a more efficient and cost-effective web search index at Yahoo. Very quickly, other groups at Yahoo started using Hadoop for research jobs and revenue-driving applications, such as click prediction for sponsored search. Since then, Hadoop has evolved to become an essential tool for many of the world’s largest companies, including Facebook, GE, Visa, and Wal-Mart.

Pinterest has big data, and it knows how to share

Pinterest is no National Security Agency, but the company, which identifies itself as a “visual discovery tool,” has grown into a collector of plenty information. Like Twitter, Facebook, Google, and other web giants, Pinterest has developed sophisticated systems for storing the data, but it’s also built a tool that lets lots of employees get at it.