Google Cloud announced that Docker and Apache Flink are now available as optional components in Dataproc. The Docker availability means you will now be able to run daemons on your Dataproc cluster nodes allowing you to interact with Hadoop clusters via your containerized applications.
The Docker component also uses Google Container Registry, in addition to the default Docker registry.
Apache Flink
Apache Beam and Apache Flink are two of the best streaming technologies today. Apache Flink is a distributed processing engine using stateful computation, while Apache Beam is a unified model for defining batch and steaming processing pipelines. Using Apache Flink as an execution engine, you can also run Apache Beam jobs on Dataproc, in addition to Google’s Cloud Dataflow service.
References
See Also
- Get Started with the new Cloud Shell Editor
- HTTP/gRPC server streaming available in Google Cloud Run
- .NET, Java and Ruby now available in Google Cloud Functions
- Eventrac, a new events functionality to build event-driven applications on the Google Cloud
- Logs Buckets and Log Views now available in the Google Cloud Platform
