Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
130
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
Hold on! You have a data team in PyCon Taiwan!
leew
0
11
朝聖之路
leew
0
65
20250328 Airflow 3.0 The First Glance
leew
0
190
踏入開源的第一步
leew
0
170
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
75
Unlocking Python's Core Magic
leew
1
260
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
170
Intro to Airflow - From Zero to Hero
leew
0
980
Does Kobayashi-San write code vulnerable to SQL injection?
leew
0
120
Other Decks in Technology
See All in Technology
Kiroから考える AIコーディングツールの潮流
s4yuba
1
340
Amazon CloudWatchのメトリクスインターバルについて / Metrics interval matters
ymotongpoo
3
280
東京海上日動におけるセキュアな開発プロセスの取り組み
miyabit
0
200
CSPヘッダー導入で実現するWebサイトの多層防御:今すぐ試せる設定例と運用知見
llamakko
1
270
ML Pipelineの開発と運用を OpenTelemetryで繋ぐ @ OpenTelemetry Meetup 2025-07
getty708
0
320
AIエージェントを支える設計
tkikuchi1002
11
2.3k
TypeScript 上達の道
ysknsid25
23
4.8k
claude codeでPrompt Engineering
iori0311
0
530
M365アカウント侵害時の初動対応
lhazy
7
5.1k
隙間時間で爆速開発! Claude Code × Vibe Coding で作るマニュアル自動生成サービス
akitomonam
2
200
DatabricksのOLTPデータベース『Lakebase』に詳しくなろう!
inoutk
0
160
VLMサービスを用いた請求書データ化検証 / SaaSxML_Session_1
sansan_randd
0
120
Featured
See All Featured
Balancing Empowerment & Direction
lara
1
510
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
138
34k
Adopting Sorbet at Scale
ufuk
77
9.5k
Six Lessons from altMBA
skipperchong
28
3.9k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Typedesign – Prime Four
hannesfritz
42
2.7k
The World Runs on Bad Software
bkeepers
PRO
70
11k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
Designing for Performance
lara
610
69k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Designing for humans not robots
tammielis
253
25k
Music & Morning Musume
bryan
46
6.7k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.