Marcin Kuthan

Big data engineer leading Allegro engineering team that develops highly scalable and fault-tolerant data ingestion and analytics platform.

GCP

FinOps for data pipelines on Google Cloud Platform

April 2, 2022 25 minute read

Do you monitor costs of the data pipelines in exactly the same way as you monitor overall health, latency or throughput? Nowadays, taking care of cost effici...

GCP Dataproc and Apache Spark tuning

March 24, 2022 8 minute read

Dataproc is a fully managed and highly scalable Google Cloud Platform service for running Apache Spark. However, “managed” does not relieve you from the prop...

GCP Cloud Composer 1.x tuning

March 15, 2022 16 minute read

I would love to only develop streaming pipelines but in reality some of them are still batch oriented. Today you will learn how to properly configure Google ...