We are excited to bring Transform 2022 back in-person July 19 and virtually July 20 - 28. Join AI and data leaders for insightful talks and exciting networking opportunities. Register today!
Google today announced that its Cloud Dataproc service — a fully managed tool based on the Hadoop and Spark open source big data software — is now generally available.
The service — which supports the MapReduce engine, the Pig platform for writing programs, and the Hive data warehousing software — first became available in beta in September. And Google has enhanced the tool since then.
This tool complements a separate service called Google Cloud Dataflow for batch and stream processing. The underlying technology for the service has been accepted as an Apache incubator project under the name Apache Beam.
Now that Google Cloud Dataproc is available for anyone to use — at 1 cent per hour for each vCPU — the Google Cloud Platform is a bit more equipped to take on public cloud market leader Amazon Web Services as well as Microsoft Azure and the IBM public cloud. All three already have big-data services.
Meanwhile there are also startups that provide Hadoop as a service. But the distinction for this service is that Cloud Dataproc hooks in with other Google cloud services, like Google Cloud Storage, Google Cloud Bigtable, and BigQuery.
Documentation for the service is here.
VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Learn more about membership.