Ex-Google, Yahoo, Facebook employees snub recession, launch Hadoop startup

October 14, 2008 | Eric Eldon | Comments |

Updated

Recession be damned, a group of top Silicon Valley engineers have come together to launch an interesting startup called Cloudera. Not yet launched, it intends to help other companies adopt a promising software platform called Hadoop.

Hadoop is an open-source software project designed to let developers write and run applications that process huge amounts of data. While it could potentially improve a wide range of other software, the ecosystem supporting its implementation is still developing. Which is where Cloudera hopes to make a place for itself.

Right in the middle of a downturn. It’s the kind of move the Valley is all about.

Cloudera will help other companies “install, configure and run” Hadoop, either on a company’s own servers or using Amazon’s hosted Elastic Compute Cloud (EC2) service.

Its founding team includes ex-Yahoo engineering vice president Amr Awadallah, ex-Google engineering star Christophe Bisciglia, former Facebook data team leader Jeff Hammerbacher and entrepreneur Mike Olson. The company hasn’t announced its investors yet, but Awadallah is currently an entrepreneur-in-residence at Facebook backer Accel Partners — so presumably that firm is already on board.

More on Hadoop: It uses the Google-introduced MapReduce systems framework that divides applications into small blocks of work, creating multiple replicas of data blocks that it places on various computer nodes. The benefits, as stated by the official Hadoop open-source site, include:

Scalable: Hadoop can reliably store and process petabytes.

Economical: It distributes the data and processing across clusters of commonly available computers. These clusters can number into the thousands of nodes.

Efficient: By distributing the data, Hadoop can process it in parallel on the nodes where the data is located. This makes it extremely rapid.

Reliable: Hadoop automatically maintains multiple copies of data and automatically redeploys computing tasks based on failures.

It is already in use at large companies like Yahoo. For a vitriolic yet informative take on the technology, see Ted Dziuba’s earlier write-up at The Register.

[Update: Kyle Shank, Dziuba's partner in crime at search engine Pressflip.com, says their startup used Hadoop at first but didn't find it ultimately relevant. He adds: "Hadoop is most useful when you need to regularly process gigabytes of information that fit the MapReduce paradigm. Cloudera will be most useful to companies that have giga/terabytes of data and no idea what to do with it." That's certainly not most companies, but it fits the description of a few companies with lots of cash.]

Next Story: Hands on with the new Apple MacBooks and Cinema Display
Previous Story: Omniture acquires Mercado assets for $6.5M

Bookmark and Share

Tags:

Photo of Eric Eldon

About the Author, Eric Eldon

Eric currently covers digital media technology and business news, especially what's happening on social networks and their platforms. He also writes and edits stories about venture capital, and lots of other stuff, too. He started at VentureBeat in the spring of 2007, half a year or so after Matt Marshall left his reporting job at the San Jose Mercury News to found the site. Eric previously cofounded a startup called Writewith, that was building editorial software for newspapers and other groups of writers. The startup didn't work out, but he learned a lot.

  • ch
    ding. thank you for playing.

    time to go back to journalism 101.

    google did not create map-reduce, nor is it a program.
  • Ding. Thanks for being a pedantry-focused hater, ch.

    To make you happy, I changed the word "created" to "introduced" and the word "program" to "systems framework."

    Feeling better now?
  • This article writes of too like. And I love you very much!
    The author was to have too much just.
    Thank author sharing.At this is once!
    UGG boots sale
    UGG boots
    Authentic Products uggsline.co.uk provides all kinds of UGG Boots
    classic short in high quality but lowest prices possible from UK.
  • Wow, that's a heck of a team... Google, Yahoo, Facebook. I would love to work in a similar team.
  • All star team, I can't wait!
  • Lot of potential here, although Hadoop in and of itself covers a lot of ground since it can be used for a lot of different purposes. Are they going to be consulting with people on how to build applications on top of Hadoop, providing toolkits for using Hadoop for specific tasks, building Hadoop-powered applications themselves, or focusing on something like HBase? Any one of those areas would be more than enough work for a single company.
  • gsmaverick
    I would take a job here any day. This is a rockstar team. Can't wait to see what they do!
  • jen
    great team, but unfortunately a super boring idea/space. helping enterprise companies setup hadoop shops over the cloud is going to be frustrating ... let alone just convincing companies to use hadoop (and how to use it). but to migrate their data to the cloud too? eek. let's see how this goes. looks niche to me.
  • This is going to be great for the Hadoop project, it has been relying too heavily on Yahoo! to carry the weight.

    One of the most common processes is the processing of log files for analytics, and there are companies processing their crawls using it, notably Yahoo! and Powerset.

    There are toolsets and such being developed inside Yahoo! for their purposes, facebook as well, supposedly all to be released as Apache projects. Most likely they will be core contributors to the project and tools, do hosting or premised based work, perform consulting, and built some integration into some proprietary systems.

    If Chris is leaving the Big G either this is a fantastic opportunity and/or things got sour for him
  • How's it a start-up? Do they intend to place IT staff in companies so they can run Hadoop? The software is already open source, isn't it? Even if they could pitch it to their CTO/CIO connections, who's gonna support it? Are they trying to become RedHat?
  • edhardy622
    UGGs became ubiquitous among Southern California surfers and Southern California downhill skiers, and from there, Uggs, which name comes from the Australian
    http://www.uggboots365.co.uk