Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Starts Airflow task execution directly from the...

Starts Airflow task execution directly from the triggerer

Lee Wei

May 08, 2024
Tweet

More Decks by Lee Wei

Other Decks in Technology

Transcript

  1. $ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__

    = [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
  2. File "weilee.py", line 1 __name__ = 李唯 / Wei Lee

    ^^^ SyntaxError: invalid syntax $ python weilee.py
  3. "execute" methods are not always needed This is the only

    logic when deferrable is set to true and does not need to be executed in the worker.
  4. How does it affect operator authors? A new way to

    implement operators in an async manner
  5. Limitations • It is necessary to go back to workers

    to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.