Upgrade to Pro — share decks privately, control downloads, hide ads and more …

KubeCon EU 2017: Dancing on the Edge of a Volcano

KubeCon EU 2017: Dancing on the Edge of a Volcano

Brandon Philips

March 31, 2017
Tweet

More Decks by Brandon Philips

Other Decks in Technology

Transcript

  1. Dancing on the Edge of a Volcano Kubernetes Security Processes

    Jess Frazelle (@jessfraz) Brandon Philips (@brandonphilips)
  2. Today's Major Parts • Context Setting • Open Source and

    Security Disclosure • Kubernetes Security Process • Helping Out
  3. Today's Major Parts • Context Setting • Open Source and

    Security Disclosure • Kubernetes Security Process • Helping Out
  4. Today's Major Parts • Context Setting • Open Source and

    Security Disclosure • Kubernetes Security Process • Helping Out
  5. An open process Need to be open to vendors and

    community but also keep information from those who can do harm before bugs are fixed.
  6. Users Expectations - Want software without bugs - Want to

    be able to upgrade without breaking changes - Want to be let know when security fixes are made so they can act on them accordingly
  7. Original Reporter Expectations - Want “a human to say acknowledged”

    - Want updates on status, transparency - Want bug to be fixed in a certain timeframe
  8. Project Expectations - Want bugs to be fixed quickly and

    embargo to be kept secret until time of disclosure - Want upgrades to remain seamless - Want to continue to instill trust with users
  9. How to get people to upgrade? - Challenges with making

    sure people know - Use mailing lists, establish a pattern for the project that allows people to follow along - How to handle if very severe… pray you don’t have someone make up a fun name
  10. How bugs get reported... I found a bug… should I

    be evil or let upstream know... Let me google “X security disclosure” or I’ll just try to email security@
  11. How bugs get reported... I’ll lookup the project’s recommended method

    for disclosures I found a bug… I’ll file an issue on the repo...
  12. I found a bug… I’ll file an issue on the

    repo... - Not everyone can recognize the severity of a bug at first glance. - Allow for mistakes and have a process for handling them in a way that it will not confuse the original reporter or cause unnecessary attention. Mistakes happen
  13. General Recommendations We Got - Make documentation Googleable (X security

    disclosure) - Have a dedicated response team - Coordinate via a mailing list - Don't force the use of GPG - Have internal documentation on the process - Use a severity leveling system - Request CVEs when appropriate*
  14. Node.js: Help the Ecosystem BP Fix the core but also

    help track the ecosystem and notify
  15. Today's Major Parts • Context Setting • Open Source and

    Security Disclosure • Kubernetes Security Process • Helping Out
  16. Kubernetes Release Process k8s 1.5.x k8s 1.6.0 k8s 1.6.x Bug

    fixes & security patches Features Features k8s 1.7.0 k8s 1.7.x Bug fixes & security patches Features
  17. Kubernetes Release Process k8s 1.5.x k8s 1.6.0 k8s 1.6.x Bug

    fixes & security patches Features Features Bug fixes & security patches ~9 Months*
  18. The Kubernetes v1.6 Release Schedule • … 4-6 weeks of

    coding … • Jan 30th (Monday) v1.6.0-alpha.1 • Feb 13th (Monday) v1.6.0-alpha.2 • Feb 20th (Monday) Cut release-1.6 branch and v1.6.0-beta.0 • Feb 27th (Monday) Start code freeze • Feb 28th (Tuesday) v1.6.0-beta.1 • March 7th (Tuesday) - v1.6.0-beta.2 • March 14th (Tuesday) Lift code freeze and v1.6.0-rc.1 • March 22nd (Wednesday) - v1.6.0
  19. The Release Team • Release Management Team Lead • Release

    Branch Manager • Docs Lead • Features Lead • Bug Triage Lead • Test Infra Lead • Automated Upgrade Testing Lead • Manual Upgrade Testing Lead • Testing Lead • Product Lead
  20. Bug report Fix team summoned Bug is reported through the

    disclosure process. within 24 hours Fix Development Disclosure of “fix forthcoming” to users within 1-14 days within 1-7 days within 1-21 days Release day Patch disclosure to distributions
  21. Bug report Fix team summoned The “Fix Team” is organized

    and a “Lead” makes sure relevant engineers are included in the discussion. within 24 hours Fix Development Disclosure of “fix forthcoming” to users within 1-14 days within 1-7 days within 1-21 days Release day Patch disclosure to distributions
  22. Bug report Fix team summoned Patches are posted to a

    private repo. They are tested and reviewed there by the “fix team.” within 24 hours Fix Development Disclosure of “fix forthcoming” to users within 1-14 days within 1-7 days within 1-21 days Release day Patch disclosure to distributions
  23. Bug report Fix team summoned “Fix Lead” emails kubernetes-announce and

    kubernetes-security-announce informing users a fix will be made available on a certain date. within 24 hours Fix Development Disclosure of “fix forthcoming” to users within 1-14 days within 1-7 days within 1-21 days Release day Patch disclosure to distributions
  24. Bug report Fix team summoned If the bug is deemed

    critical enough, downstream distributions will be given notice about the bug and patches ahead of time. within 24 hours Fix Development Disclosure of “fix forthcoming” to users within 1-14 days within 1-7 days within 1-21 days Release day Patch disclosure to distributions
  25. Bug report Fix team summoned Release is cut from the

    private repo and patches to the public repo are made after binaries are available. within 24 hours Fix Development Disclosure of “fix forthcoming” to users within 1-14 days within 1-7 days within 1-21 days Release day Patch disclosure to distributions
  26. Why is Heartbleed only a 5.0? CVSS 2.0 scoring is

    failing to take into account quite a few factors: 1. It’s a target of opportunity for attackers: 30-70% of the Internet was effected 2. It’s being actively and successfully exploited on the Internet 3. It’s easy to exploit
  27. Rotating the Responsibilities - Challenges - Responsible security process requires

    non-public work - Running the process is not glamorous - Incentives - Many organizations rely on Kubernetes; the work is important - Critical security announcement will be highly referenced - Next Steps - We need a way of making decisions! Kubernetes Governance Discussion concludes BP
  28. Communication to Users - Send an email to kubernetes-announce and

    call it a day?! - How do you make announcements immediately actionable? - Should vendors be informed ahead of time with a fix? 1 JF
  29. Today's Major Parts • Context Setting • Open Source and

    Security Disclosure • Kubernetes Security Process • Helping Out
  30. Some Homework - Join Kubernetes Announce! - Review the Kubernetes

    Security Process - Governance discussion to figure out - Help improve https://github.com/kubernetes/test-infra