Upgrade to Pro — share decks privately, control downloads, hide ads and more …

KubeCon 2016: Self-hosted, Scale, and Federatio...

KubeCon 2016: Self-hosted, Scale, and Federation with Kubernetes v1.4 and Beyond

Brandon Philips

November 08, 2016
Tweet

More Decks by Brandon Philips

Other Decks in Programming

Transcript

  1. Brandon Philips @brandonphilips | [email protected] | coreos.com Kubernetes v1.4 and

    Beyond Self-hosted, Scale, and Federation Stickers at the Front
  2. 90+ Projects on GitHub, 1,000+ Contributors OPEN SOURCE Secure solutions,

    support plans, training + more ENTERPRISE CoreOS is Running the World’s Containers Linux
  3. etcd Overview • Introduced in 2013 by CoreOS • Primary

    datastore of Kubernetes • Replicated consistent key/value • Revisions and event stream • Auto-leader election for availability
  4. - etcd v2.0 could support ~1000 node - Snapshot: ~200MB

    - etcd v3.0 tested on ~3000 node + history - Snapshot: 1GB+ - Testing underway for 5000+ nodes Other parts of Kubernetes are scaling limit now etcd v3 + Kubernetes API
  5. - etcd v2.0 could support ~1000 node - Snapshot: ~200MB

    - etcd v3.0 tested on ~3000 node + history - Snapshot: 1GB+ - Testing underway for 5000+ nodes Other parts of Kubernetes are scaling limit now etcd v3 + Kubernetes API
  6. etcd Operator • Operates etcd clusters on k8s • Handles

    common tasks: ◦ Resize ◦ Upgrade ◦ Backup ◦ Healing • Configured by resource
  7. go // golang source code package main import "os" func

    main() { compile_go(os.Args[1:]) } go
  8. - Use Kubernetes compute, net, storage abstractions and APIs -

    Let infrastructure tools focus on infrastructure - Foundation to automate cluster upgrades Self-hosting Goals
  9. Kubernetes Running Kubernetes $ kubectl -n kube-system get daemonsets NAME

    DESIRED CURRENT NODE-SELECTOR AGE kube-apiserver 2 2 master=true 4d kube-proxy 3 3 <none> 4d $ kubectl -n kube-system get deployments NAME DESIRED CURRENT UP-TO-DATE kube-controller-manager 1 1 1 kube-scheduler 2 2 2
  10. API Server • API server is state of Kubernetes •

    Designed for single failure domain • Can handle 1000s of machines
  11. Federation API Server • Holds state for multiple clusters •

    Spans multiple failure domains • Coordinates between clusters not for clusters
  12. LoadBalancer federation-apiserver New York San Francisco federation-controller Federation Control Plane

    A: A: etcd federation-apiserver federation-controller etcd LoadBalancer kubernetes-federation.example.com A: A:
  13. Federation Next Steps • Try out the federation setup guides

    ◦ Kelsey's works easier than the official guides • Get involved with SIG Federation
  14. • Make self-hosted default for Kubernetes clusters • Easy to

    operate etcd for single and federated • Bring federation to market on these technologies My Goals from Here