Apache Kafka - A Complete Engineer's Guide

Apache Kafka is a distributed event streaming platform originally built at LinkedIn to solve the challenge of routing millions of events per second across complex data pipelines. This deck walks through its five core primitives — Producers, Topics, Partitions, Consumers, and Brokers — and explains how architectural decisions like sequential I/O and zero-copy transfers deliver unmatched throughput. A side-by-side comparison with RabbitMQ and Amazon Kinesis helps engineers decide when Kafka is the right tool for the job.

Transcript