Upgrade to Pro — share decks privately, control downloads, hide ads and more …

State of the RubyGems 2023

State of the RubyGems 2023

Given at RubyConf 2023 in San Diego, CA.

Samuel E. Giddins

November 14, 2023
Tweet

More Decks by Samuel E. Giddins

Other Decks in Programming

Transcript

  1. Your intrepid presenter @segiddins → Samuel Giddins → RubyGems, Bundler,

    RubyGems.org maintainer → Security lead → 10+ year bug contributor 2 — State of the RubyGems 2023 @ RubyConf
  2. Where did we come from? RubyGems was created at this

    very conference 19 years ago 3 — State of the RubyGems 2023 @ RubyConf
  3. Where did we come from? Growth of the Ruby ecosystem

    made it hard to maintain many gems on a system by hand 4 — State of the RubyGems 2023 @ RubyConf
  4. Where did we come from? Bundler came in 2009 to

    solve the problem of maintaining gems by hand. Increased usage made it difficult for volunteers to keep up with the growing number of Bundler & RubyGems.org users 5 — State of the RubyGems 2023 @ RubyConf
  5. RubyGems statistics As of October 2023, our major numbers are

    → 181,745 users → 192,825 gems → 1,555,458 versions of gems → 147,326,326,048 total gem downloads → average 20,000 requests/ second → average 2 billion requests/weekday → maximum 225,000 requests/second → 7.5 TB/hour, 185 TB/day → 4.6 PB/month, 54 PB/year 9 — State of the RubyGems 2023 @ RubyConf
  6. Ruby Central’s Open Source team → a 24/7 on-call team

    → the SRE team, configuring and operating Fastly, AWS, Kubernetes, Postgres, OpenSearch, Redis, and more → the RubyGems.org team, maintaining the Rails application → the RubyGems team, maintaining the gem command → the Bundler team, maintaining the bundle command → the Ruby Toolbox team → the RubyAPI team → the Gemstash team 12 — State of the RubyGems 2023 @ RubyConf
  7. Ruby Central’s Open Source team Thank you to each &

    every contributor 13 — State of the RubyGems 2023 @ RubyConf
  8. Ruby Central’s Open Source team Here at RubyConf Colby Martin

    14 — State of the RubyGems 2023 @ RubyConf
  9. What have we done? Kept the lights on 16 —

    State of the RubyGems 2023 @ RubyConf
  10. What have we done? Made many releases 17 — State

    of the RubyGems 2023 @ RubyConf
  11. What have we done? Lots of good stuff 18 —

    State of the RubyGems 2023 @ RubyConf
  12. Running RubyGems is challenging Incidents → Dependency API deprecation caused

    a 25x increase in traffic, from ~10k rps to ~225k rps → Migrating from unicorn to puma (and renaming the k8s deployment) lost our horizontal pod autoscaler rule, causing the single pod to get overwhelmed → Early May pager storm 20 — State of the RubyGems 2023 @ RubyConf
  13. Running RubyGems is challenging Incidents Weekly AWS costs Weekly bytes

    served by the Rails app 21 — State of the RubyGems 2023 @ RubyConf
  14. Running RubyGems is challenging Recurring security issues → Dependency confusion

    → Account takeover → Good maintainer gone bad 22 — State of the RubyGems 2023 @ RubyConf
  15. Running RubyGems is challenging Service uptime → DDoS → Bitrot

    → New security vulns → Deprecated infrastructure 23 — State of the RubyGems 2023 @ RubyConf
  16. Running RubyGems is challenging Support requests → Lost account access

    → Abandoned (sometimes empty) projects → Limited supply of names for gems → Higher security means more customer service →Requiring MFA means many more manual MFA resets by staff → Ownership disputes →Multiple people claim ownership →Companies claim trademark or IP 24 — State of the RubyGems 2023 @ RubyConf
  17. What about 2023? Improved MFA and added support for passkeys

    instead of OTP codes 28 — State of the RubyGems 2023 @ RubyConf
  18. What about 2023? Better uptime and stability by removing the

    dependency API 29 — State of the RubyGems 2023 @ RubyConf
  19. What about 2023? → Implemented a PubGrub-based dependency resolver for

    Bundler → Added gem exec to RubyGems → Shipped the bundle compose beta 30 — State of the RubyGems 2023 @ RubyConf
  20. What about 2023? Merged hundreds of pull requests across 19

    versions of Bundler & RubyGems → Significant performance improvements for large application bundles → ruby(file: ".ruby-version") support for Gemfiles to re-use version files → fully allowlist-based safe loading for Marshal files, completely removing a repeated source of security issues 31 — State of the RubyGems 2023 @ RubyConf
  21. What about 2023? Made maintaining RubyGems.org significantly easier → Continued

    migration to infrastructure as code → Multiple environments for testing changes → Web-based admin tools to reduce SSH & console access → Backend support for gem contents and OIDC auth → User-facing features based on these improvements coming soon 32 — State of the RubyGems 2023 @ RubyConf
  22. What about 2023? It's been a productive year! 33 —

    State of the RubyGems 2023 @ RubyConf
  23. How much does this cost? AWS $20,000 / month 36

    — State of the RubyGems 2023 @ RubyConf
  24. How much does this cost? Fastly $500,000 / month 37

    — State of the RubyGems 2023 @ RubyConf
  25. How much does this cost? Developers $45,000 - $55,000 /

    month 38 — State of the RubyGems 2023 @ RubyConf
  26. Funding sources Donated services → DNSimple → DataDog → Honeybadger

    → Mend.io → AWS → Fastly 41 — State of the RubyGems 2023 @ RubyConf
  27. Funding sources Memberships rubycentral.org → Single developers → Small companies

    → Large corporations 44 — State of the RubyGems 2023 @ RubyConf
  28. What we fund right now 24/7/365 On-Call Rotation 47 —

    State of the RubyGems 2023 @ RubyConf
  29. What we fund right now Infrastructural work → SRE work

    → Gem transparency 48 — State of the RubyGems 2023 @ RubyConf
  30. What we fund right now Industry-wide collaboration → Languages →

    Companies 49 — State of the RubyGems 2023 @ RubyConf
  31. What we fund right now Gem clients → RubyGems →

    Bundler 50 — State of the RubyGems 2023 @ RubyConf
  32. What we fund right now Slow & steady progress 51

    — State of the RubyGems 2023 @ RubyConf
  33. We would like to fund more 52 — State of

    the RubyGems 2023 @ RubyConf
  34. Our goals Full-time security role → Trusted publishing → Checksum

    verification → Full passkey support → TUF (the update framework) → SLSA-compliant builders → Sigstore support 54 — State of the RubyGems 2023 @ RubyConf
  35. Our goals Ecosystem quality-of-life improvements → More gem information →

    Browsable gem contents → Downloads over time → Hosted & searchable documentation → In-browser gem playgrounds 55 — State of the RubyGems 2023 @ RubyConf
  36. How to support us Read RFCs and give us feedback.

    github.com/rubygems/rfcs 58 — State of the RubyGems 2023 @ RubyConf
  37. How to support us Share this info 59 — State

    of the RubyGems 2023 @ RubyConf
  38. How to support us Become a member rubycentral.org 60 —

    State of the RubyGems 2023 @ RubyConf
  39. How to support us Become a partner to Ruby Central

    Adarsh [email protected] 61 — State of the RubyGems 2023 @ RubyConf