Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
99
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
20250328 Airflow 3.0 The First Glance
leew
0
55
踏入開源的第一步
leew
0
140
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
52
Unlocking Python's Core Magic
leew
1
230
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
140
Intro to Airflow - From Zero to Hero
leew
0
920
Does Kobayashi-San write code vulnerable to SQL injection?
leew
0
100
Back to the last time we were here
leew
0
29
Atomic Commits - An Easy & Proven Way to Manage & Automate Release Process
leew
0
440
Other Decks in Technology
See All in Technology
AWSLambdaMCPServerを使ってツールとMCPサーバを分離する
tkikuchi
1
3k
ビジネスとデザインとエンジニアリングを繋ぐために 一人のエンジニアは何ができるか / What can a single engineer do to connect business, design, and engineering?
kaminashi
0
100
4/16/25 - SFJug - Java meets AI: Build LLM-Powered Apps with LangChain4j
edeandrea
PRO
2
120
QA/SDETの現在と、これからの挑戦
imtnd
0
140
AI Agentを「期待通り」に動かすために:設計アプローチの模索と現在地
kworkdev
PRO
2
460
PagerDuty×ポストモーテムで築く障害対応文化/Building a culture of incident response with PagerDuty and postmortems
aeonpeople
1
320
コスト最適重視でAurora PostgreSQLのログ分析基盤を作ってみた #jawsug_tokyo
non97
0
410
白金鉱業Meetup_Vol.18_AIエージェント時代のUI/UX設計
brainpadpr
1
150
地味にいろいろあった! 2025春のAmazon Bedrockアップデートおさらい
minorun365
PRO
1
270
読んで学ぶ Amplify Gen2 / Amplify と CDK の関係を紐解く #jawsug_tokyo
tacck
PRO
1
160
2025-04-24 "Manga AI Understanding & Localization" Furukawa Arata (CyberAgent, Inc)
ornew
1
200
Mastraに入門してみた ~AWS CDKを添えて~
tsukuboshi
0
270
Featured
See All Featured
Side Projects
sachag
452
42k
We Have a Design System, Now What?
morganepeng
52
7.5k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Building a Modern Day E-commerce SEO Strategy
aleyda
40
7.2k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
5
550
Building Flexible Design Systems
yeseniaperezcruz
329
38k
Writing Fast Ruby
sferik
628
61k
Faster Mobile Websites
deanohume
306
31k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.1k
StorybookのUI Testing Handbookを読んだ
zakiyama
29
5.6k
Embracing the Ebb and Flow
colly
85
4.6k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.