VentureBeat

Doug Cutting, the respected open source engineer who has led the way in speeding up the processing of huge amounts of data, is leaving Yahoo to join a startup called Cloudera.

Cutting announced the move on his blog yesterday. The move is significant for two reasons. One, it’s one more sign that Yahoo is losing the guts of its search intelligence now that it has ceded its search initiatives to Microsoft leadership (even though Cutting says he’s not leaving in protest). Second, at Cloudera, it brings in one place a number of engineers with deep experience with Hadoop, the open source platform founded by Cutting designed to process huge amounts of data quickly. Cloudera is seeking to commercialize the Hadoop technology. The company is focused on making it accessible for a wider group of users than just expert developers. The Hadoop technology is widely considered the most powerful and cost-effective way to manage large amounts of data, store it, and derive intelligence from it.

Already at Cloudera are people like Mike Olson, former chief executive of Sleepy Cat, the open source embedded database engine that was acquired by Oracle; Amr Awadallah, who earlier led Yahoo’s efforts to improve search; Jeff Hammerbacher, a former Facebook employee who led the company’s data and machine learning technology initiatives; and Christophe Bisciglia, who formerly led Google’s cloud computing initiatives and worked on Google’s core data processing technologies, MapReduce and GFS.

Hadoop was inspired by Google’s MapReduce, a framework that cut new ground by dividing applications into smaller blocks of work. Those blocks, or nodes, would do this again, allowing each end node to process a smaller problem and then pass the answers back to the master node. The difference is that while MapReduce is kept closed by Google, Hadoop is made freely available to anyone. The trick for Cloudera is to offer Hadoop in a way that is easily packaged and downloadable, and to offer services around it, just as other companies seeking to sell open source software have sought to do — with mixed results.

Cloudera, based in San Francisco, raised $6 million in funding in June. Backers include Accel Partners, former VMware chief executive Diane Greene, former MySQL chief executive Marten Mickos, and Facebook chief financial officer Gideon Yu.

Cutting will join Cloudera in September.

Update: Here’s an excerpt of correspondence I just had with chief executive Olson:

Doug’s going to continue working on the Avro sub-project for Hadoop — work he’s been doing for some time now, and work that Cloudera has been supporting with other engineers already on staff. It’s an important new piece of critical infrastructure in Hadoop, and we are glad to see it coming along quickly.

More generally, Doug’s going to work on the open source core projects, where his influence has long been substantial. He’ll also participate in planning and executing Cloudera’s product roadmap. His standing in the community, his excellent design sense and his experience in building large-scale distributed systems will be valuable to us.

Next Story: BP gives nod to algae biofuels with $10M to Martek
Previous Story: Google’s new search update “Caffeine” changes both look and feel

Tags: co:cloudera, people:doug cutting