Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Event-Driven Applications: Apache Kafka and Python
Search
FTisiot
March 24, 2021
Technology
0
950
Event-Driven Applications: Apache Kafka and Python
How to create Event-Driven Applications with Apache Kafka and Python
FTisiot
March 24, 2021
Tweet
Share
More Decks by FTisiot
See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
100
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
120
Build an Open Source Data Pipeline
ftisiot
0
290
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
710
Fix Your Strings in PostgreSQL
ftisiot
0
260
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
210
Track Database Changes with Apache Kafka
ftisiot
0
270
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
940
Kickstart your Kafka with Faker Data
ftisiot
0
410
Other Decks in Technology
See All in Technology
stupid jj tricks
indirect
0
7.9k
Modern_Data_Stack最新動向クイズ_買収_AI_激動の2025年_.pdf
sagara
0
190
バイブコーディングと継続的デプロイメント
nwiizo
2
400
職種別ミートアップで社内から盛り上げる アウトプット文化の醸成と関係強化/ #DevRelKaigi
nishiuma
2
130
インサイト情報からどこまで自動化できるか試してみた
takas0522
0
140
AWSにおけるTrend Vision Oneの効果について
shimak
0
120
データエンジニアがこの先生きのこるには...?
10xinc
0
440
研究開発部メンバーの働き⽅ / Sansan R&D Profile
sansan33
PRO
3
20k
From Prompt to Product @ How to Web 2025, Bucharest, Romania
janwerner
0
110
「技術負債にならない・間違えない」 権限管理の設計と実装
naro143
35
11k
o11yで育てる、強い内製開発組織
_awache
3
110
空間を設計する力を考える / 20251004 Naoki Takahashi
shift_evolve
PRO
3
320
Featured
See All Featured
The Cost Of JavaScript in 2023
addyosmani
53
9k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Building an army of robots
kneath
306
46k
Site-Speed That Sticks
csswizardry
11
880
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3.1k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.5k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.1k
Context Engineering - Making Every Token Count
addyosmani
5
170
It's Worth the Effort
3n
187
28k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Documentation Writing (for coders)
carmenintech
75
5k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Transcript
Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Event Driven Applications
@ftisiot What is an Event?
@ftisiot Why Event-driven?
@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3
@ftisiot Distributed Brokers Replication Factor 3 2
@ftisiot Distributed Replication Factor 2
@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding
@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3
@ftisiot Pizza Demo!
@ftisiot Log Size
@ftisiot Topic Partitions
@ftisiot Distributed
@ftisiot Topic Retention Policies Time Size
@ftisiot Selecting a Partition <K, V> Same Key Same Partition
@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition
@ftisiot Scaling Out P1 P2 P3 C1 C2
@ftisiot Partitions Demo!
@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2
@ftisiot Consumer Groups Demo!
@ftisiot Kafka Connect Kafka Connect Kafka Connect
@ftisiot Evolve Existing Applications Kafka Connect CDC
@ftisiot Distribute Events Config File Only!
@ftisiot Kafka Connect Demo!
@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
@ftisiot Ask me Anything