Setup EKS Multi-cluster using Federation v2

Transcript

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Setup EKS Multi-cluster using Federation v2 Kyle Bai Co-organizer Cloud Native Taiwan User Group Cloud Native Taiwan User Group
2. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. @k2r2bai About Me ⽩白凱仁(Kyle Bai) • Software Engineer at inwinSTACK. • OSS Contributor. • Certified Kubernetes Administrator. • Co-organizer of Cloud Native Taiwan User Group. • Interested in emerging technologies. GitHub: kairen(k2r2.bai@gmail.com) Blog: https://k2r2bai.com
3. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. @k2r2bai Agenda Today I would like to talk about • Motivations • Introducing KubeFed(Federation V2) • Demo • Summary
4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. @k2r2bai S U M M I T Motivations
5. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data Centers Networking Servers Application Storage Virtualization OS Hardware Accelerator Drivers Database Runtime Application OS Data Centers Networking Servers Application Storage Virtualization OS Laptop Enterprise IT Public Cloud Customer Managed Provider Managed Database Runtime Database Runtime Drivers Drivers
6. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data Centers Networking Servers Application Storage Virtualization OS Hardware Accelerator Drivers Database Runtime Application OS Data Centers Networking Servers Application Storage Virtualization OS Laptop Enterprise IT Public Cloud Customer Managed Provider Managed Database Runtime Database Runtime Drivers Drivers
7. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data Centers Networking Servers Application Storage Virtualization OS Hardware Accelerator Drivers Database Runtime Application OS Data Centers Networking Servers Application Storage Virtualization OS Customer Managed Provider Managed Database Runtime Database Runtime Drivers Drivers Laptop Enterprise IT Public Cloud
8. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data Centers Networking Servers Application Storage Virtualization OS Hardware Accelerator Drivers Database Runtime Application OS Data Centers Networking Servers Application Storage Virtualization OS Laptop (Dev) Enterprise IT (Staging) Public Cloud (Production) Customer Managed Provider Managed Database Runtime Database Runtime Drivers Drivers
9. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Dev Staging Production
10. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Dev Staging Production US EU AP …
11. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Dev Staging Production US EU AP …
12. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. TW Local Dev
13. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. TW Local Dev TW US Staging
14. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. TW Local Dev TW US Staging AP … US EU Production
15. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. But… How make it easy to manage clusters and resources?
16. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cluster Federation Clusters Users UI CLI API Federation Control Plane Resource Resource Resource Container Resource Resource Resource Container Region / Availability Zone
17. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cluster Federation Federation makes it easy to manage multiple Kubernetes clusters. • Sync resources across clusters: Federation provides the ability to keep resources in multiple clusters in sync. • Cross cluster discovery: Federation provides the ability to auto-configure DNS servers and load balancers with backends from all clusters.
18. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Benefit of Federation • Sensitive Workloads: I have multiple clusters but want to run sensitive workloads only in specific clusters. • High availability: Single region outage does not impact the availability of workloads. • Avoiding provider lock-in: By making it easier to migrate applications across clusters, federation prevents cluster provider lock-in. • Hybrid Cloud: Extend Deployments from on-premise clusters to the cloud.
19. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Benefit of Applications • Distribution of applications, services, and policy to multiple clusters. • Migration of applications and services and their storage between clusters • Disaster recovery for those applications and services. • Serving users from clusters closest to them.
20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. @k2r2bai S U M M I T Introducing KubeFed(aka Federation V2)
21. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Concepts KubeFed is configured with two types of information: • Cluster configuration declares which clusters KubeFed should target. • Type configuration declares which API types KubeFed should handle. • Templates • Placement • Overrides
22. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sync controller kubefedctl federate <resource> (autogenerate typeConfig and type CRDs) kubefedctl join/unjoin <cluster> Propagation refers to how resources are distributed to the target clusters.
23. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. kubefedctl federate configmap FederatedConfigMap ConfigMap Create con\gmap type conXguration Set FederatedCon\gMap to manage Con\gMap
24. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Type Configuration - Templates Templates define the representation of a resource common across clusters.
25. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Type Configuration - Placement Placement defines which clusters the resource is intended to appear in.
26. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Type Configuration - Overrides Overrides define per-cluster field-level variation to apply to the template.
27. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Higher Order Behaviour • Scheduling refers to a decision-making capability that can decide how workloads should be spread across different clusters similar to how a human operator would. • Multi-Cluster DNS provides the ability to programmatically manage DNS resource records of Kubernetes Service or Ingress objects.
28. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scheduling Manager SchedulingPreference Controller ServiceDNS Controller IngressDNS Controller DNSEndpoint Controller
29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. @k2r2bai S U M M I T Demo
30. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Set up Federation Cluster
31. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Demo
32. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Demo • Use Federated API to deploy an application across Kubernetes clusters. • Use Multi-Cluster DNS API to automatically sync DNS resources records in supported DNS providers(Route53). • Use Scheduling API to constrain(or maintain) the number of replicas for application.
33. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
34. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ServiceDNSRecord Object DNSEndpoint Controller Watch/List CRUD DNSEndpoint Object ExternalDNS Controller CRUD Service DNS Controller Watch/List Watch/List Cluster A Cluster N Sync Watch/List DNS Provider Update status
35. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ReplicaSchedulingPreference Object CRUD SchedulingPreference Controller FederatedDeployment Object Sync Controller Watch/List Modify .spec.overrides Watch/List ap-no`heast us-east us-west Deployment Object Deployment Object Deployment Object Modify .spec.replicas totalReplicas: 15 clusters: "*": weight: 2 maxReplicas: 12 ap-noaheast: minReplicas: 1 maxReplicas: 3 weight: 1 replicas=3 replicas=6 replicas=6
36. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ReplicaSchedulingPreference Object CRUD SchedulingPreference Controller FederatedDeployment Object Sync Controller Watch/List Modify .spec.overrides Watch/List ap-no`heast us-east us-west Deployment Object Deployment Object Deployment Object Modify .spec.replicas totalReplicas: 15 clusters: "*": weight: 2 maxReplicas: 12 ap-noaheast: minReplicas: 1 maxReplicas: 3 weight: 1 replicas=3 replicas=0 replicas=12
37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. @k2r2bai S U M M I T Summary
38. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Summary • Federation v2 uses CustomResourceDefinitions to extend Kubernetes with new APIs. • The building blocks approach allows the extension of federation to supported and custom resources, which only enhances the flexibility Federation v2 is providing for future development. • Although Federation v2 is in the prototype stage, I believe that the community behind the Federation V2 project is strong and that the project seems to be headed in the right direction.
39. @k2r2bai S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. References • https://github.com/kubernetes-sigs/federation-v2 • https://blog.openshift.com/combining-federation-v2-and-istio-multicluster/ • https://blog.openshift.com/kubernetes-federation-v2-on-openshift-3-11/ • https://medium.com/condenastengineering/k8s-federation-v2-a-guide-on-how-to- get-started-ec9cc26b1fa7 • https://kubernetes.io/blog/2018/12/12/kubernetes-federation-evolution/ • https://static.sched.com/hosted_files/kccna18/d4/ SIG%20MultiCluster%20Deep%20dive%20at%20Kubecon%20Seattle%202018.pdf
40. Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Kyle Bai k2r2.bai@gmail.com