Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
120
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
朝聖之路
leew
0
58
20250328 Airflow 3.0 The First Glance
leew
0
160
踏入開源的第一步
leew
0
160
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
69
Unlocking Python's Core Magic
leew
1
260
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
160
Intro to Airflow - From Zero to Hero
leew
0
960
Does Kobayashi-San write code vulnerable to SQL injection?
leew
0
120
Back to the last time we were here
leew
0
36
Other Decks in Technology
See All in Technology
エンジニア向け技術スタック情報
kauche
1
250
Amazon Bedrockで実現する 新たな学習体験
kzkmaeda
1
510
Prox Industries株式会社 会社紹介資料
proxindustries
0
270
IIWレポートからみるID業界で話題のMCP
fujie
0
780
製造業からパッケージ製品まで、あらゆる領域をカバー!生成AIを利用したテストシナリオ生成 / 20250627 Suguru Ishii
shift_evolve
PRO
1
130
生成AIで小説を書くためにプロンプトの制約や原則について学ぶ / prompt-engineering-for-ai-fiction
nwiizo
3
820
Amazon ECS & AWS Fargate 運用アーキテクチャ2025 / Amazon ECS and AWS Fargate Ops Architecture 2025
iselegant
16
5.3k
AWS CDK 実践的アプローチ N選 / aws-cdk-practical-approaches
gotok365
6
700
本当に使える?AutoUpgrade の新機能を実践検証してみた
oracle4engineer
PRO
1
140
データプラットフォーム技術におけるメダリオンアーキテクチャという考え方/DataPlatformWithMedallionArchitecture
smdmts
5
620
Абьюзим random_bytes(). Фёдор Кулаков, разработчик Lamoda Tech
lamodatech
0
330
Understanding_Thread_Tuning_for_Inference_Servers_of_Deep_Models.pdf
lycorptech_jp
PRO
0
110
Featured
See All Featured
Automating Front-end Workflow
addyosmani
1370
200k
Building Adaptive Systems
keathley
43
2.6k
Code Review Best Practice
trishagee
68
18k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
16
940
Why Our Code Smells
bkeepers
PRO
337
57k
Facilitating Awesome Meetings
lara
54
6.4k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.3k
RailsConf 2023
tenderlove
30
1.1k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
The World Runs on Bad Software
bkeepers
PRO
69
11k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.