Upgrade to Pro — share decks privately, control downloads, hide ads and more …

piwheels: a Python package repository for Raspb...

piwheels: a Python package repository for Raspberry Pi

Talk given at remote python pizza April 2020

Ben Nuttall

April 25, 2020
Tweet

More Decks by Ben Nuttall

Other Decks in Technology

Transcript

  1. @ben_nuttall Ben Nuttall • Software engineer at BBC News Labs

    • Formerly at Raspberry Pi Foundation • Creator of gpiozero, piwheels and pyjokes • Opensource.com correspondent • twitter.com/ben_nuttall • github.com/bennuttall
  2. @ben_nuttall Why piwheels? • Before piwheels, “pip install” on Raspberry

    Pi would take ages • People don’t generally build Arm platform wheels • Just x86/x86_64 for Linux, Mac & Windows • I wondered if it would be possible to build a repository to host Arm wheels • It is • So I did
  3. @ben_nuttall Uploading Python package with C extensions to PyPI Python

    version 2.7 3.5 3.6 3.7 3.8 Windows (win32) Windows (amd64) Mac OS (x86_64) Linux (i686) Linux (x86_64) Source (sdist)
  4. @ben_nuttall What is piwheels? • Open source project maintained by:

    • Ben Nuttall • Dave Jones • Tooling to automate building wheels of everything on PyPI • Wheels are pre-compiled binary distributions of Python packages • Tailored towards the Arm platform • linux_armv6l and linux_armv7l platform tags
  5. @ben_nuttall What is piwheels.org? • Python package repository (like PyPI)

    • Hosts Arm wheels of all packages on PyPI • Website comprises: • Simple index • Project pages • Blog & info pages
  6. @ben_nuttall The stack • Master • Distributes jobs, web server

    hosts index and wheel files • Postgres database • Monitors PyPI, generates build queue • Logs build attempts, logs downloads • Builders • Get given build jobs • 3 × Raspbian Jessie (Python 3.4) • 2 × Raspbian Stretch (Python 3.5) • 2 × Raspbian Buster (Python 3.7) • Easy to scale up/down
  7. @ben_nuttall The stack • Master • Distributes jobs, web server

    hosts index and wheel files • Postgres database • Monitors PyPI, generates build queue • Logs build attempts, logs downloads • Builders • Get given build jobs • 3 × Raspbian Jessie (Python 3.4) • 2 × Raspbian Stretch (Python 3.5) • 2 × Raspbian Buster (Python 3.7) • Easy to scale up/down
  8. @ben_nuttall Using piwheels • Raspbian is pre-configured to use piwheels.org

    as an additional index to PyPI • Users get platform wheels without knowing it’s there • Tutorials that say "warning: this step will take ages" are now wrong
  9. @ben_nuttall Using piwheels • Raspbian is pre-configured to use piwheels.org

    as an additional index to PyPI • Users get platform wheels without knowing it’s there • Tutorials that say "warning: this step will take ages" are now wrong
  10. @ben_nuttall How many downloads? • 2018: 5 million • 2019:

    9 million • 2020 so far: 4 million • To date: 19 million • March 2020: 1.2 million • Yesterday: 40,000
  11. @ben_nuttall How much time saved? • 2018: 42 years •

    2019: 128 years • 2020 so far: 76 years • To date: 250 years • March 2020: 22 years • Yesterday: 262 days
  12. @ben_nuttall What’s next for piwheels? • Implement new PEPs •

    Requires-Python • Yank • Deploy JSON API • Package information at piwheels.org/project/<project>/json • Keep fixing broken builds • Get ready for Debian Bullseye (Python ~3.9) in 2021