Event-Driven Applications: Apache Kafka and Python

Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot

@ftisiot Why Kafka?

@ftisiot Event Driven Applications

@ftisiot What is an Event?

@ftisiot Why Event-driven?

@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3

@ftisiot Distributed Brokers Replication Factor 3 2

@ftisiot Distributed Replication Factor 2

@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >

@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding

@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3

@ftisiot Pizza Demo!

@ftisiot Log Size

@ftisiot Topic Partitions

@ftisiot Distributed

@ftisiot Topic Retention Policies Time Size

@ftisiot Selecting a Partition <K, V> Same Key Same Partition

@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition

@ftisiot Scaling Out P1 P2 P3 C1 C2

@ftisiot Partitions Demo!

@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2

@ftisiot Consumer Groups Demo!

@ftisiot Kafka Connect Kafka Connect Kafka Connect

@ftisiot Evolve Existing Applications Kafka Connect CDC

@ftisiot Distribute Events Config File Only!

@ftisiot Kafka Connect Demo!

@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks

@ftisiot Ask me Anything

Event-Driven Applications: Apache Kafka and Python

Event-Driven Applications: Apache Kafka and Python

FTisiot

More Decks by FTisiot

Other Decks in Technology

Featured

Transcript