Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Keeping the Gems Sparkling

Keeping the Gems Sparkling

Samuel E. Giddins

October 30, 2024
Tweet

More Decks by Samuel E. Giddins

Other Decks in Technology

Transcript

  1. 1. Entertain you all, who make this ecosystem so vibrant

    2. Shock you as much as possible with big numbers 3. Give you slides to send to your boss as fuel to ask for more support for our work Too honest? Sorry. I'll just lean into #1 then 😅 Goals Samuel Giddins 4
  2. 1. Bring industry best practices into the Ruby ecosystem 2.

    Keep Ruby a vibrant community used by everyone, from the hobbyist to the Fortune 500 3. Convince you to do things that make me look successful to my boss 4. See midnight sun Goals Samuel Giddins 5
  3. 10 - 11 million unique IPs per month 23,302,340,473 requests

    per month 20,000 peak Requests per Second 167,947,962,107 gem downloads RubyGems.org Stats Samuel Giddins 10
  4. Story Time aka the weekend sam got paged aka the

    time we hit 225k rps Samuel Giddins 12
  5. Big Stuff Trusted Publishing OIDC GitHub Actions for secure gem

    build & push Bundler Checksums (in alpha) importmaps + Stimulus.js What have we been up to? Samuel Giddins 16
  6. Small Stuff We removed avatars from rubygems.org …for privacy (using

    gravatar leaks user emails) then we put them back (proxied through rubygems.org) Up to 8x the speed for gem installs and pristine What have we been up to? Samuel Giddins 18
  7. Deploying (Dis)continuous Manual Deployment Multiple times per day (or week

    if I forget...) Zero Downtime Absolutely Zero, no questions asked, no excuses, end of discussion 0-locking database migrations & upgrades Watch for more info from @simi for how he pulled this off repeatedly with PGBouncer and RDS What have we been up to? Samuel Giddins 20
  8. 24x7 Oncall Follow the sun Coordinating between Australia, India, Czech

    Republic, Spain, USA Sometimes staying up late & waking up early so we get some face time What have we been up to? Samuel Giddins 21
  9. Supported by our generous sponsors AWS gives us $165,000/year in

    credits (offsetting infrastructure costs) Fastly gives us $1,000,000/year in donated services (estimated at retail rate) DataDog donates monitoring services Honeybadger provides error tracking All this is... $$$$$ Samuel Giddins 22
  10. Supported by donations to Ruby Central Shopify $1 million over

    4 years to support the security & reliability of RubyGems & RubyGems.org Directly funds Open Source team work German Sovereign Technology Fund €863,000 over about 2 years Funded general maintenance & security-focused improvements All this is... $$$$$ Samuel Giddins 23
  11. Supported by donations to Ruby Central OpenSSF Alpha-Omega $100,000 security

    audit $150,000 to add organizations to RubyGems.org AWS Credits Sponsor my role as Security Engineer in Residence Pay for me to be here today! All this is... $$$$$ Samuel Giddins 24
  12. 5 engineers on staff All part-time except me (thanks AWS!)

    This is a second job for most of my teammates All this requires work Samuel Giddins 25
  13. A close-knit support team from other companies @hsbt @ AndPad

    & Ruby Core Eric * Jenny @ Shopify Maciej @ Mend.io All this requires work Samuel Giddins 26
  14. Individual contributors like YOU! We receive support, feedback, and Pull

    Requests from all around the Ruby Community! Thank you! All this requires work Samuel Giddins 27
  15. Expertise is hard-won I've been working on Bundler & RubyGems

    for 10 years We want to retain our staff Financial support ebbs and flows All this requires work Samuel Giddins 28
  16. New versions of gems (& Rails) RubyGems.org is one continually

    maintained app since 2009 Started as a Sinatra app on Ruby 1.8 Converted to Rails 2, then 3, 4, 5, 6, and 7 A canonical open source Rails app Infrastructure always falls behind Samuel Giddins 32
  17. New best practices Don't use that, use this sprockets propshaft

    webpacker importmaps sass tailwind Infrastructure always falls behind Samuel Giddins 33
  18. Keeping up with RubyGems & Bundler improvements RubyGems.org compact index

    makes dependency resolution faster Developed in coordination with RubyGems.org Infrastructure always falls behind Samuel Giddins 34
  19. Governments now care about this. See: The US Government saying

    C is bad Constant security concerns Samuel Giddins 37
  20. The security of software used by the Government is vital

    to the Government’s ability to perform its critical functions. The development of commercial software often lacks transparency, sufficient focus on the ability of the software to resist attack, and adequate controls to prevent tampering by malicious actors. There is a pressing need to implement more rigorous and predictable mechanisms for ensuring that products function securely, and as intended. Samuel Giddins 38
  21. The security and integrity of “critical software” — software that

    performs functions critical to trust (such as affording or requiring elevated system privileges or direct access to networking and computing resources) — is a particular concern. Accordingly, the Government must take action to rapidly improve the security and integrity of the software supply chain, with a priority on addressing critical software. Samuel Giddins 39
  22. Like it or not, our random RubyGems have been critical

    commercial software, according to the US Government How do we deal with this reality? Constant security concerns Samuel Giddins 40
  23. 430 HackerOne reports Responsible disclosure & bug bounty program Each

    report takes time to triage Some reports require a lot of testing 64 real issues fixed So, so, so many reports that are... useless Constant security concerns Samuel Giddins 41
  24. CISA, US Dept Homeland Security Coordination on publishing of Principles

    for Package Repository Security Constant security concerns Samuel Giddins 42
  25. Starting the process of a 3rd-party audit Scoping work Staying

    within a very limited budget Coordinating on verification of remediation Constant security concerns Samuel Giddins 43
  26. Attacks ? Hey, why is the site getting so many

    500s right now? Why am I getting paged? Why is there one IP making 10k requests per second? Why is the site down? Oh, a security researcher is literally hitting every endpoint for every gem on the whole system Added a missing index, asked the researcher to use the DB dumps instead Constant security concerns Samuel Giddins 45
  27. RubyGems Research Every gem, every file, indexed Full-text search File-level

    diffs between versions Fast response to xz Search every gem for anything related to xz or liblzma Constant security concerns Samuel Giddins 46
  28. Sigstore sign. verify. protect. Making sure your software is what

    it claims to be. Improvements Samuel Giddins 47
  29. Trusted Publishing Automate publishing gems from CI No more persistent

    credentials No more 2fa dance Improvements Samuel Giddins 48
  30. Trusted Publishing Come set it up for __your__ gems at

    my campfire this afternoon! Improvements Samuel Giddins 49
  31. This is all made possible by contribution from users like

    you, your companies, and security-minded organizations like the German government (STF), OpenSSF (Alpha-Omega), AWS, Shopify, and more. Samuel Giddins 50
  32. This talk is made possible by the support of my

    wife, who has allowed me to spend the week after our wedding in Scandinavia. Samuel Giddins 51