Cloud Dataproc is probably one of the lesser-known products in Google Cloud’s portfolio, but it’s a powerful tool for data wranglers who are looking for a fully managed cloud service that lets them run Apache Spark and Hadoop clusters without having to worry about managing the underlying infrastructure. Today. Google announced that it is launching the alpha of Cloud Dataproc to Kubernetes — and while that, too, may not sound all that interesting at first, it’s an important step for Google Cloud as it works to adapt more of its products to a hybrid cloud model.
The general idea here is to give enterprise customers (and make no mistake, enterprise customers are the main focus of Google Cloud these days) the ability to run Apache Spark jobs on Google Kubernetes Engine (GKE) clusters. With products like Anthos now making GKE available virtually anywhere, this means customers can now also take Cloud Dataproc to their own data centers. Right now, the service only supports Apache Spark, but Google plans to support other open-source projects, too.
“Enterprises are increasingly looking for products and services that support data processing across multiple locations and platforms,” said Matt Aslett, Research Vice President at 451 Research. “The launch of Cloud Dataproc on Kubernetes is significant in that it provides customers with a single control plane for deploying and managing Apache Spark jobs on Google Kubernetes Engine in both public cloud and on-premises environments.”
Typically, Spark applications run on Hadoop YARN clusters. Google notes that the Cloud Dataproc on Kubernetes will free users from having to use two cluster management systems and will give them a single view across both YARN and Kubernetes clusters. “Supporting both YARN and Kubernetes can bring your enterprise the needed flexibility to modernize certain hybrid workloads while continuing to monitor YARN-based workloads,” the company writes in today’s announcement.
The new service is now available as an alpha. If you want to give it a try, you’ll have to apply for access by emailing Google.
Read Full Article
No comments:
Post a Comment