Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Catch in Rye: Seeding Change and Lessons Le...

The Catch in Rye: Seeding Change and Lessons Learned

A presentation at EuroPython 2024 about Rye and Python packaging.

Armin Ronacher

July 11, 2024
Tweet

More Decks by Armin Ronacher

Other Decks in Programming

Transcript

  1. Who am I? Armin @mitsuhiko Ronacher Things you might know

    I worked on: Flask, Werkzeug, Jinja, Pygments, Sphinx, LogBook, itsdangerous, Click, MarkupSafe, Sentry, Babel, …
  2. So what did I do? Poured a lot of time

    into Sentry Started to enjoy the green pastures of Rust
  3. Rust A language empowering everyone to build reliable and ef

    fi cient software. • Let's not kid ourselves: it's bloody complicated • Yet as a programmer you're surprisingly productive with it • The ecosystem has excellent DX • The language values backwards compatibility • The language values innovation and progress
  4. Going back in Time (~2014) • I picked up Rust

    properly when I used Python 2 actively • Cargo was not yet a thing • Python 3 was in a state of very slow and painful adoption
  5. The Zen of Python “There should be one — and

    preferably only one — obvious way to do it.”
  6. I Saw The Light • Packaging doesn't have to be

    painful • Downloading the compiler/interpreter doesn't have to be painful • Switching between compiler/interpreter versions can be trivial • One can have the same experience on Linux, macOS, and Windows
  7. • The only goal is to dominate • If it

    does not dominate, something else should • “I just want it solved”
  8. • I do not work for Astral • I gave

    Rye's stewardship to Astral • uv — today — is a replacement for pip-tools/pip/venv • uv tomorrow will fully replace the need of Rye by absorbing it in spirit
  9. Does it work? • Yes, but there are issues •

    Many of the issues are not technical challenges
  10. So what the the challenges? • Dev Dependencies • Local

    Dependencies • Workspaces • pyproject.toml Limitations (PEP 508) • Single Version Resolution • Good Python Builds
  11. Dev Dependencies • Every Tool invents dev dependencies • Some

    could bene fi t from isolation • black, ru ff , … • Others do not work with isolation • pytest, … • Others are mixed • fl ake8, …
  12. Dev Dependencies • There is no standard, everyone invents one

    • Potential solution: • reserve a "dev" extra group • add a "tool" dependency group?
  13. Local Dependencies • What about temporary overrides? • What about

    editable installs? • No standard relative path URL syntax • Potential solution: adjacent con fi g to override packages
  14. Workspaces • But they don't work well yet • They

    are Rye proprietary • Again run into challenges with relative paths
  15. pyproject.toml Limitations • Dependency string array is too limiting •

    Where do you store dependency attached meta information? • Impossible to encode even more into these strings without breaking already existing tools • Who can write these strings?
  16. pyproject.toml Limitations • Why do you need meta information? •

    Pick the right index (PyPI vs internal) • Git checkout, local paths, multi-version matches • Tool speci fi c proprietary (even if only temporary) extra information
  17. Other Issues with pyproject.toml • Dynamic metadata is a bad

    idea • Already countless of proprietary extensions by di ff erent tools • Many di ff erent ways to de fi ne licenses • Complex resolutions caused by markers
  18. Portable Locking • rye/uv support experimental universal locking • it

    does not yet have a stable and supported cross platform lock format • The problem is “not easy”
  19. Single Version Resolution • a >= 1 • a <

    1 • How can you ever fi nd a solution? • Rust/Node: permits multi-version resolutions • Issues: sys.modules (though solvable), C-extension modules (CABI)
  20. Good Python Builds • We need PEP 711: PyBI: a

    standard format for distributing Python Binaries • indygreg builds are great, but have portability issues (bad CFLAGS, missing readline, …) • Not an o ff i cial project, run by a single person