Изучаем облачные решения Google по работе с ML, BIG Data
Cloud Dataproc is a managed Spark and Hadoop service that lets you take advantage of open source data tools for batch processing, querying, streaming, and machine learning. Cloud Dataproc automation helps you create clusters quickly, manage them easily, and save money by turning clusters off when you don’t need them. With less time and money spent on administration, you can focus on your jobs and your data.
Apache Beam on Cloud Dataflow provides autoscaling data pipelines.