Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Event-Driven Applications: Apache Kafka and Python
Search
FTisiot
March 24, 2021
Technology
0
950
Event-Driven Applications: Apache Kafka and Python
How to create Event-Driven Applications with Apache Kafka and Python
FTisiot
March 24, 2021
Tweet
Share
More Decks by FTisiot
See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
110
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
130
Build an Open Source Data Pipeline
ftisiot
0
290
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
720
Fix Your Strings in PostgreSQL
ftisiot
0
260
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
220
Track Database Changes with Apache Kafka
ftisiot
0
270
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
950
Kickstart your Kafka with Faker Data
ftisiot
0
410
Other Decks in Technology
See All in Technology
GraphRAG グラフDBを使ったLLM生成(自作漫画DBを用いた具体例を用いて)
seaturt1e
1
150
Amazon Athena で JSON・Parquet・Iceberg のデータを検索し、性能を比較してみた
shigeruoda
1
150
OPENLOGI Company Profile for engineer
hr01
1
45k
猫でもわかるAmazon Q Developer CLI 解体新書
kentapapa
1
120
会社を支える Pythonという言語戦略 ~なぜPythonを主要言語にしているのか?~
curekoshimizu
3
880
コンパウンド組織のCRE #cre_meetup
layerx
PRO
1
280
AI時代の開発を加速する組織づくり - ブログでは書けなかったリアル
hiro8ma
2
330
AWS re:Invent 2025事前勉強会資料 / AWS re:Invent 2025 pre study meetup
kinunori
0
660
RemoteFunctionを使ったコロケーション
mkazutaka
1
130
What's new in OpenShift 4.20
redhatlivestreaming
0
320
Azure Well-Architected Framework入門
tomokusaba
1
140
AIの個性を理解し、指揮する
shoota
1
230
Featured
See All Featured
We Have a Design System, Now What?
morganepeng
53
7.8k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
21
1.2k
GitHub's CSS Performance
jonrohan
1032
470k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
140
34k
Visualization
eitanlees
149
16k
Navigating Team Friction
lara
190
15k
[RailsConf 2023] Rails as a piece of cake
palkan
57
5.9k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
249
1.3M
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Building a Modern Day E-commerce SEO Strategy
aleyda
44
7.8k
How GitHub (no longer) Works
holman
315
140k
Transcript
Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Event Driven Applications
@ftisiot What is an Event?
@ftisiot Why Event-driven?
@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3
@ftisiot Distributed Brokers Replication Factor 3 2
@ftisiot Distributed Replication Factor 2
@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding
@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3
@ftisiot Pizza Demo!
@ftisiot Log Size
@ftisiot Topic Partitions
@ftisiot Distributed
@ftisiot Topic Retention Policies Time Size
@ftisiot Selecting a Partition <K, V> Same Key Same Partition
@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition
@ftisiot Scaling Out P1 P2 P3 C1 C2
@ftisiot Partitions Demo!
@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2
@ftisiot Consumer Groups Demo!
@ftisiot Kafka Connect Kafka Connect Kafka Connect
@ftisiot Evolve Existing Applications Kafka Connect CDC
@ftisiot Distribute Events Config File Only!
@ftisiot Kafka Connect Demo!
@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
@ftisiot Ask me Anything