Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Catch in Rye: Seeding Change and Lessons Le...

The Catch in Rye: Seeding Change and Lessons Learned

A presentation at EuroPython 2024 about Rye and Python packaging.

Avatar for Armin Ronacher

Armin Ronacher

July 11, 2024
Tweet

More Decks by Armin Ronacher

Other Decks in Programming

Transcript

  1. Who am I? Armin @mitsuhiko Ronacher Things you might know

    I worked on: Flask, Werkzeug, Jinja, Pygments, Sphinx, LogBook, itsdangerous, Click, MarkupSafe, Sentry, Babel, …
  2. So what did I do? Poured a lot of time

    into Sentry Started to enjoy the green pastures of Rust
  3. Rust A language empowering everyone to build reliable and ef

    fi cient software. • Let's not kid ourselves: it's bloody complicated • Yet as a programmer you're surprisingly productive with it • The ecosystem has excellent DX • The language values backwards compatibility • The language values innovation and progress
  4. Going back in Time (~2014) • I picked up Rust

    properly when I used Python 2 actively • Cargo was not yet a thing • Python 3 was in a state of very slow and painful adoption
  5. The Zen of Python “There should be one — and

    preferably only one — obvious way to do it.”
  6. I Saw The Light • Packaging doesn't have to be

    painful • Downloading the compiler/interpreter doesn't have to be painful • Switching between compiler/interpreter versions can be trivial • One can have the same experience on Linux, macOS, and Windows
  7. • The only goal is to dominate • If it

    does not dominate, something else should • “I just want it solved”
  8. • I do not work for Astral • I gave

    Rye's stewardship to Astral • uv — today — is a replacement for pip-tools/pip/venv • uv tomorrow will fully replace the need of Rye by absorbing it in spirit
  9. Does it work? • Yes, but there are issues •

    Many of the issues are not technical challenges
  10. So what the the challenges? • Dev Dependencies • Local

    Dependencies • Workspaces • pyproject.toml Limitations (PEP 508) • Single Version Resolution • Good Python Builds
  11. Dev Dependencies • Every Tool invents dev dependencies • Some

    could bene fi t from isolation • black, ru ff , … • Others do not work with isolation • pytest, … • Others are mixed • fl ake8, …
  12. Dev Dependencies • There is no standard, everyone invents one

    • Potential solution: • reserve a "dev" extra group • add a "tool" dependency group?
  13. Local Dependencies • What about temporary overrides? • What about

    editable installs? • No standard relative path URL syntax • Potential solution: adjacent con fi g to override packages
  14. Workspaces • But they don't work well yet • They

    are Rye proprietary • Again run into challenges with relative paths
  15. pyproject.toml Limitations • Dependency string array is too limiting •

    Where do you store dependency attached meta information? • Impossible to encode even more into these strings without breaking already existing tools • Who can write these strings?
  16. pyproject.toml Limitations • Why do you need meta information? •

    Pick the right index (PyPI vs internal) • Git checkout, local paths, multi-version matches • Tool speci fi c proprietary (even if only temporary) extra information
  17. Other Issues with pyproject.toml • Dynamic metadata is a bad

    idea • Already countless of proprietary extensions by di ff erent tools • Many di ff erent ways to de fi ne licenses • Complex resolutions caused by markers
  18. Portable Locking • rye/uv support experimental universal locking • it

    does not yet have a stable and supported cross platform lock format • The problem is “not easy”
  19. Single Version Resolution • a >= 1 • a <

    1 • How can you ever fi nd a solution? • Rust/Node: permits multi-version resolutions • Issues: sys.modules (though solvable), C-extension modules (CABI)
  20. Good Python Builds • We need PEP 711: PyBI: a

    standard format for distributing Python Binaries • indygreg builds are great, but have portability issues (bad CFLAGS, missing readline, …) • Not an o ff i cial project, run by a single person