The companies will create big data centers that researchers can tap for internet-wide research. The initiative will promote collaboration among industry, academia and government by removing the financial and logistical barriers to research in data-intensive, large scale computing. Researchers can use the internet-wide testing environment to explore cloud-computing software, data center management and hardware needs.
“We will foster a sense of an intellectual commons, where we all contribute and benefit from a pipe that is larger,” said Prabhakar Raghavan, (pictured left) head of Yahoo Research, in a conference call. “It’s about what kinds of interesting projects you can do, once you take the cloud for granted, as if it were a utility.”
While everyone will benefit from the test bed, the research will clearly pit Intel, HP and Yahoo against IBM and Google, which have their own cloud-computing research underway. Andrew Chien, vice president at Intel Research (pictured at bottom), said that this effort is broader in terms of its availability to researchers. It also allows for research on all levels, not just on the application level as Google and IBM are doing.
The companies have partnered with the Infocomm Development Authority of Singapore (IDA), the University of Illinois at Urbana-Champaign, and the Karlsruhe Institute of Technology (KIT) in Germany to form the research initiative. The partnership with Illinois also includes the National Science Foundation.
The test bed will initially consist of six research centers at IDA, the University of Illinois at Urbana-Champaign, the Steinbuch Centre for Computing of the Karlsruhe Institute of Technology, HP Labs, Intel Research and Yahoo. Each location will host a cloud computing infrastructure. The machines will use HP hardware and Intel processors — 1,000 to 4,000 processor cores. The test bed locations are expected to be available to researchers worldwide through a selection process later this year. Prith Banerjee, head of HP Labs (pictured left), said each company would contribute researchers and data centers.
HP said it will focus on five research projects. One, dubbed “cells as a service,” is a prototype management system for the cloud infrastructure. It creates “service cells,” or secure self-contained virtual infrastructure, that are isolated from the rest of a cloud. Such cells can expand as needed and be automated. Banerjee also said HP is working on another project to cut the costs and improve the service of future data centers.
The test bed will run Apache Hadoop –- an open source, distributed computing project of the Apache Software Foundation -– and other open source, distributed computing software such as Pig, the parallel programming language developed by Yahoo Research.
HP Labs recently sharpened its focus to help HP and its customers capitalize on the industry’s shift toward cloud computing, a driving force behind HP’s vision of Everything as a Service. Under that vision, internet devices and services will interact seamlessly through the cloud, and businesses and individuals will use services that anticipate their needs based on location, preferences, calendar and communities.
In November 2007, Yahoo! announced a data center, called M45, for cloud computing research; Carnegie Mellon University was the first institution to take advantage of this supercomputer. Yahoo also announced this year an agreement with Computational Research Laboratories (CRL) to jointly support cloud computing research and make one of the 10 fastest supercomputers in the world available to academic institutions in India.