Cluster Autoscaler: Dynamic & Automatic Reconfiguration

Presented at Kubernetes Meetup Tokyo #6: https://k8sjp.connpass.com/event/60324/

1. Cluster Autoscaler Dynamic & Automatic Reconfiguration @mumoshu, primary maintainer of kube-aws/contributor of cluster- autoscaler Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 1
2. Կͷ࿩ • K8S on AWSͰΦʔτεέʔϦϯάΛ͢Δ • pod͡Όͳͯ͘nodeͷ΄͏ • cluster-autoscaler(CA)Λ࢖͏ Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 2
3. Cluster Autoscaler: FAQ • Ϋϥ΢υϓϩόΠμඪ४ͷΦʔτεέʔϦϯάͱͷҧ͍? • AWS AutoScaling: based on CPU/RequestCount(ALB) • Cluster Autoscaler: Autoscaling based on total resource requests • e.g. add node(s) when CA sees pending pods • Pod࡞Γ͗ͯᷓ͢ΕͨΒϊʔυ૿΍͢Α! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 3
4. Why: I was too lazy to operate CA • K8SΫϥελͷӡ༻΋΍Διϑτ΢ΣΞΤϯδχΞ • Φʔτεέʔϧ͸ར༻͍ͨ͠ • CAͷӡ༻͕஍ຯʹΊΜͲ͍͘͞ I'm responsible to run several production K8S clusters on AWS. I'd like to enable autoscaling on them but I'm too lazy to operate CA. I'm basically a software who loves sketching & coding, not operating ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 4
5. Why: Syncing node pool config between cloud<->CA • ASG on AWSͱCA on K8Sͷઃఆ͸ผ • Cloud Provider: Node Pool Min/Max Size • CA: Node Pool Min/Max Size • Change(s) on AWS-side requires updating/restarting CA • ໃ६͕͋Δͱਖ਼͘͠ಈ͔ͳ͍ • AWSଆͰઃఆΛม͑ͨΒɺCAͷઃఆΛม͑ͯ࠶ىಈ͕ඞཁ Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 5
6. Node Pools/ϊʔυϓʔϧ • AutoScaling Group(AWS) aws autoscaling update-auto-scaling-group \ --auto-scaling-group-name $NAME \ --min-size $MIN \ --max-size $MAX • Instance Group in GCE? Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 6
7. CA 1. kubectl edit deployment cluster-autoscaler 2. cluster-autoscaler --nodes $MIN:$MAX:$NAME Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 7
8. 2౓खؒ • ࡞ۀ΋Εͦ͠͏ • Scripting? • ͜Ε͚ͩͷͨΊʹ…? Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 8
9. ASGͱCAͷઃఆ • Ͳ͕ͬͪਖ਼ͳͷ? • ASGͷ৔߹ • CAͷmin >= ASGͷmin • CAͷmax <= ASGͷmax Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 9
10. How • ASGͷઃఆΛCA͕ࣗಈతʹಡΜͰ͘ΕΕ͹Α͘ͳ͍? Shouldn't CA automatically sync up with AWS about ASGs it is told to manage? Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 10
11. What • CAͷ৽ػೳ 1. Dynamic Reconfiguration 2. Node Group Automatic Discovery ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 11
12. Dynamic Reconfiguration • CAͷઃఆΛConfigMapʹॻ͘ • ConfigMapΛॻ͖׵͑ΔͱCA͕ࣗಈϦϩʔυ(--configmap) • PR: github.com/kubernetes/contrib/pull/2226 Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 12
13. Dynamic Reconfigurationͷར༻ํ๏ kubectl create configmap cluster-autoscaler -f ca-config.yaml Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 13
14. kind: ConfigMap apiVersion: v1 metadata: name: cluster-autoscaler namespace: kube-system data: settings: |- { "nodeGroups": [ { "minSize": 1, "maxSize": 2, "name": "myasg" } ] } Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 14
15. ./cluster-autoscaler --configmap=cluster-autoscaler ... Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 15
16. όʔδϣϯ • CA 0.5.2͔Β Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 16
17. Dynamic ReconfigurationͷͭΒ͍ͱ͜Ζ • kubectl edit deployment ca͕kubectl edit configmap caʹมΘ͚ͬͨͩͰ͸ • ͔֬ʹएׯ៉ྷʹεΫϦϓτԽ͸͠΍͘͢ͳΔ͚Ͳ • BEFORE: εΫϦϓτ͸deploymentͷ໊લͱશઃఆɺcaί ϚϯυϥΠϯͷॻ͖׵͑ํΛ஌Βͳ͍ͱ͍͚ͳ͍ • AFTER: configmapͷ໊લͱઃఆ஋͚ͩ஌͍ͬͯΕ͹͍͍ Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 17
18. ͦ͜Ͱ Node Group Automatic Discovery Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 18
19. Automatic discovery in nutshell ҰݴͰ͍͏ͱɺ CA͕AWS APIΛ࢖ͬͯASGͷઃఆΛࣗಈಉظͯ͘͠ΕΔػೳ cluster-autoscaler: Re: AWS Autoscaler autodiscover ASG names and sizes github.com/kubernetes/autoscaler/pull/11 Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 19
20. How it works ͋ͳͨ: • ASGʹλάΛ͚ͭΔ • CAʹλάΛڭ͑Δ CA: • λάͷ͍ͭͨASGΛݕ஌͢Δ ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 20
21. How operating CA improved? BEFORE: cluster-autoscaler --nodes 1:10:myasg-123456 & ASGઃఆมߋͷͨͼʹ͜ͷҾ਺ΛରԠͤ͞Δӡ༻ AFTER: cluster-autoscaler --node-group-automatic-discovery asg:tag:key1=val1,key2=val2 & ӡ༻ͷखؒͳ͠ ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 21
22. ࠓͷͱ͜ΖAWSͷΈ • ΠϯλϑΣʔε͸GCE΍Azureʹ΋ରԠͰ͖ΔΑ͏ʹ੾ͬͨ ͭ΋Γ The automatic discovery feature works only with the AWS provider as of today. However, the common interface is there for you to easily add GCE/Azure supports ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 22
23. GCEରԠΠϝʔδ AWS(Supported): --cloudprovider=aws \ --node-group-automatic-discovery asg:tag:key1=val1,key2=val2 GCE(Unsupported/Imaginary interface): --cloudprovider=gce \ --node-group-automatic-discovery gce:label:key1=val1,key2=val2 An instance group could be "labeled" in GCE, right? Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 23
24. GCE, AzureରԠٻΉ! Would anyone mind contributing supports for GCE and Azure to the automatic discovery feature?! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 24
25. Supported since CA 0.6 • Automatic Discovery͸CA 0.6͔Β • kube-aws v0.9.8-rc.1͔Β • CA 0.6͕σϑΥϧτ • Automatic DiscoveryΛΑ͠ͳʹઃఆͯ͘͠ΕΔ Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 25
26. kube-aws: Configuring CA autoscaling: clusterAutoscaler: enabled: true Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 26
27. kube-aws: Configuring CA per node pool worker: nodePools: # CA is disabled by default - name: static # CA is enabled only on this node pool - name: dynamic autoscaling: clusterAutoscaler: enabled: true Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 27
28. ·ͱΊ/Recap • CAͷӡ༻ࣗಈԽ/ָʹΦʔτεέʔϦϯά͕࢖͑Δ • ͍·ͷͱ͜ΖAWS޲͚ͷํ͕ॆ࣮ • Automate a boring operational task for CA • Dynamic Reconfiguration: supported since CA 0.5.2 • Automatic Discovery: supported on AWS since CA 0.6 • kube-aws supports CA & Automatic Discovery Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 28