Marcin Kuthan

Big data engineer leading Allegro engineering team that develops highly scalable and fault-tolerant data ingestion and analytics platform.

Apache Kafka

Kafka Streams DSL vs processor API

November 2, 2017 29 minute read

Kafka Streams is a Java library for building real-time, highly scalable, fault-tolerant, distributed applications. The library is fully integrated with Kaf...

Spark application assembly for cluster deployments

March 11, 2016 12 minute read

When I tried to deploy my first Spark application on a YARN cluster, I realized that there was no clear and concise instruction how to prepare the applicati...

Spark and Kafka integration patterns - part 2

January 29, 2016 21 minute read

In the world beyond batch, streaming data processing is a future of dig data. Despite of the streaming framework using for data processing, tight integrati...

Spark and Kafka integration patterns - part 1

August 6, 2015 less than 1 minute read

I published post on the allegro.tech blog, how to integrate Spark Streaming and Kafka. In the blog post you will find how to avoid java.io.NotSerializableExc...