Google launches Cloud Dataproc service out of beta

Google today announced that its Cloud Dataproc service — a fully managed tool based on the Hadoop and Spark open source big data software — is now generally available.

The service — which supports the MapReduce engine, the Pig platform for writing programs, and the Hive data warehousing software — first became available in beta in September. And Google has enhanced the tool since then.

“While in beta, Cloud Dataproc added several important features including property tuning, VM metadata and tagging, and cluster versioning,” Google product manager James Malone wrote in a blog post.

This tool complements a separate service called Google Cloud Dataflow for batch and stream processing. The underlying technology for the service has been accepted as an Apache incubator project under the name Apache Beam.

Now that Google Cloud Dataproc is available for anyone to use — at 1 cent per hour for each vCPU — the Google Cloud Platform is a bit more equipped to take on public cloud market leader Amazon Web Services as well as Microsoft Azure and the IBM public cloud. All three already have big-data services.

Meanwhile there are also startups that provide Hadoop as a service. But the distinction for this service is that Cloud Dataproc hooks in with other Google cloud services, like Google Cloud Storage, Google Cloud Bigtable, and BigQuery.

Documentation for the service is here.