2017 JFokus - Managing Cloud Native Applications with Kubernetes - End to End

Slides from JFokus 2017 3.5hr tutorial session: https://www.jfokus.se/jfokus/talks.jsp#ManagingCloudNativeA

The presentation is mostly composed of live demos and thus most of the slides were not used.

1. @saturnism @googlecloud @kubernetesio @jfokus Managing Cloud Native Application with Kubernetes End-to-End
2. @saturnism @googlecloud @kubernetesio @jfokus Time Topics 09:00 - 10:30 Containers, Orchestration Scheduling, Kubernetes 10:30 - 11:00 Break. YAY! 11:00 - 12:30 Kubernetes Federation CI/CD
3. @saturnism @googlecloud @kubernetesio @jfokus Ray Tsang Developer Advocate Google Cloud Platform @saturnism | +RayTsang
4. @saturnism @googlecloud @kubernetesio @jfokus Ray Tsang Developer Architect Traveler Photographer flickr.com/saturnism
5. @saturnism @googlecloud @kubernetesio @jfokus
6. @saturnism @googlecloud @kubernetesio @jfokus Cloud Native? != Running in the Cloud
7. @saturnism @googlecloud @kubernetesio @jfokus Cloud Native? Innovate fast Iterate fast Focus on core business differentiators
8. @saturnism @googlecloud @kubernetesio @jfokus Cloud Native? Infrastructure - elastic, automated Environment - consistent, agile Application - scalable, resilient, service oriented Deployment - continuous Everything - measurable
9. @saturnism @googlecloud @kubernetesio @jfokus Cloud Native? Infrastructure - elastic, automated Environment - consistent, agile Application - scalable, resilient, service oriented Deployment - continuous Everything - measurable DevOps Machines as Cattles Containerization Orchestration Microservices CI/CD Orchestration Trace, Monitoring Metrics
10. @saturnism @googlecloud @kubernetesio @jfokus
11. @saturnism @googlecloud @kubernetesio @jfokus Hello World Service - Greet Guestbook Service - Create Guestbook Service - Retrieve
12. @saturnism @googlecloud @kubernetesio @jfokus Guestbook UI Hello World Service Redis session replication greeting MySQL Guestbook Service CRUD
13. @saturnism @googlecloud @kubernetesio @jfokus Package & Deployment application.ear app.war helloworld-service.jar guestbook-service.jar apache-xyz.jar application.war /... helloworld-service.jar guestbook-service.jar apache-xyz.jar
14. @saturnism @googlecloud @kubernetesio @jfokus Machine app.war / app.ear Application Server Kernel Shell / CLI / Tools
15. @saturnism @googlecloud @kubernetesio @jfokus Machine app.war / app.ear Application Server Kernel Shell / CLI / Tools Machine app.war / app.ear Application Server Kernel Shell / CLI / Tools Machine app.war / app.ear Application Server Kernel Shell / CLI / Tools
16. @saturnism @googlecloud @kubernetesio @jfokus Let’s see write some code
17. @saturnism @googlecloud @kubernetesio @jfokus Microservices helloworld-service.jar guestbook-service.jar app.jar
18. @saturnism @googlecloud @kubernetesio @jfokus Deployment? Just run it! java -jar helloworld-service.jar java -jar guestbook-service.jar java -jar app.jar
19. @saturnism @googlecloud @kubernetesio @jfokus So many services Deploy, Manage, Bin Pack, Ports, Discovery, Isolation… How?
20. @saturnism @googlecloud @kubernetesio @jfokus Challenges Which host machine(s) should each service run on? Avoid port conflicts? How does the application code find the Guest Book service? How do we keep all of these service running? What happens if a host machine has trouble? Are the services healthy? How do we scale when load changes? Run this in another environment? QA, dev, another cloud, your servers?
21. @saturnism @googlecloud @kubernetesio @jfokus Containers To the Rescue!
22. @saturnism @googlecloud @kubernetesio @jfokus Old Way: Shared machines kernel libs app app app No isolation No namespacing Common libs Highly coupled apps and OS app
23. @saturnism @googlecloud @kubernetesio @jfokus Old Way: Virtual machines Some isolation Expensive and inefficient Still highly coupled to the guest OS Hard to manage app libs kernel libs app app kernel app libs libs kernel kernel
24. @saturnism @googlecloud @kubernetesio @jfokus New Way: Containers libs app kernel libs app libs app libs app
25. @saturnism @googlecloud @kubernetesio @jfokus Containerize Option #1 Dockerfile
26. @saturnism @googlecloud @kubernetesio @jfokus Containerize Option #2 spotify/docker-maven-plugin mvn docker:build or, spotify/dockerfile-maven-plugin or, fabric8 maven plugin
27. @saturnism @googlecloud @kubernetesio @jfokus Containerize Option #3 Docker Hub / GitHub saturnism/spring-boot
28. @saturnism @googlecloud @kubernetesio @jfokus Static Binary! Runs anywhere
29. @saturnism @googlecloud @kubernetesio @jfokus More on Containers Container Basics
30. @saturnism @googlecloud @kubernetesio @jfokus Let’s run the container! docker run -ti -p 8080:8080 helloworld-service
31. @saturnism @googlecloud @kubernetesio @jfokus Docker Compose docker-compose up
32. @saturnism @googlecloud @kubernetesio @jfokus Versioning container image docker tag spring-boot-demo spring-boot-demo:1.0
33. @saturnism @googlecloud @kubernetesio @jfokus Versioning container image docker build -t spring-boot-demo:${BUILDNO}
34. @saturnism @googlecloud @kubernetesio @jfokus Don’t Log to Container Filesystem! Log to a volume… docker -v /tmp/log:/log Or Send it elsewhere! STDOUT is captured by Docker / Kubernetes
35. @saturnism @googlecloud @kubernetesio @jfokus Clean up disk spaces Every image, layer, and, even containers litters docker rm $(docker ps -a -q) docker rmi $(docker images -q --filter dangling=true)
36. @saturnism @googlecloud @kubernetesio @jfokus Combine RUN commands apt-get update && apt-get install xyz && apt-get clean Saves you space.
37. @saturnism @googlecloud @kubernetesio @jfokus Consider Slim JARs Base Layer - java:8 (not frequently updated) Next Layer - Dependency JARs (not frequently updated) Last Layer - Application JAR (frequently updated) Use dockerfile-maven-plugin, copy-dependencies or slimfast
38. @saturnism @googlecloud @kubernetesio @jfokus Run with --rm docker run -ti --rm saturnism/spring-boot-helloworld-service:1.0
39. @saturnism @googlecloud @kubernetesio @jfokus Everything at Google runs in containers Launch over 2 billion containers per week.
40. @saturnism @googlecloud @kubernetesio @jfokus
41. @saturnism @googlecloud @kubernetesio @jfokus Control Plane Apps Software Engineers
42. @saturnism @googlecloud @kubernetesio @jfokus job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks } 10000
43. @saturnism @googlecloud @kubernetesio @jfokus web browsers BorgMaster link shard UI shard BorgMaster link shard UI shard BorgMaster link shard UI shard BorgMaster link shard UI shard Scheduler borgcfg web browsers scheduler Borglet Borglet Borglet Borglet Config file BorgMaster link shard UI shard persistent store (Paxos) Binary What just happened?
44. @saturnism @googlecloud @kubernetesio @jfokus Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Image by Connie Zhou Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world!
45. @saturnism @googlecloud @kubernetesio @jfokus
46. @saturnism @googlecloud @kubernetesio @jfokus Containers Orchestration
47. @saturnism @googlecloud #kubernetes #devoxx
48. @saturnism @googlecloud #kubernetes #devoxx Kubernetes Greek for “Helmsman”; also the root of the words “governor” and “cybernetic” • Infrastructure for containers • Schedules, runs, and manages containers on virtual and physical machines • Platform for automating deployment, scaling, and operations • Inspired and informed by Google’s experiences and internal systems • 100% Open source, written in Go
49. @saturnism @googlecloud @kubernetesio @jfokus web browsers Scheduler kubectl web browsers scheduler Kubelet Kubelet Kubelet Kubelet Config file Kubernetes Master Container Image Developer View What just happened?
50. @saturnism @googlecloud @kubernetesio @jfokus Developer View spec: containers: - name: myservice image: myservice resources: limits: memory: "128Mi" cpu: "0.1" ports: - containerPort: 3306 protocol: TCP replicas: 1 10000
51. @saturnism @googlecloud @kubernetesio @jfokus Cluster of Machines as One
52. @saturnism @googlecloud @kubernetesio @jfokus Live Demo
53. @saturnism @googlecloud #kubernetes #devoxx
54. @saturnism @googlecloud @kubernetesio @jfokus
55. @saturnism @googlecloud @kubernetesio @jfokus Hello World Service - Greet Guestbook Service - Create Guestbook Service - Retrieve
56. @saturnism @googlecloud @kubernetesio @jfokus Guestbook UI Hello World Service Redis session replication greeting MySQL Guestbook Service CRUD
57. @saturnism @googlecloud @kubernetesio @jfokus Staging vs. production Use Namespaces - deploy in the same infrastructure
58. @saturnism @googlecloud @kubernetesio @jfokus Canary Use service, and replication controllers to canary new versions
59. @saturnism @googlecloud @kubernetesio @jfokus Rollback Super simple with versioned containers
60. @saturnism @googlecloud @kubernetesio @jfokus Persistent Volumes
61. @saturnism @googlecloud @kubernetesio @jfokus Rolling upgrade Similar to canary, but slowly let the new version take over
62. @saturnism @googlecloud @kubernetesio @jfokus Health Checks
63. @saturnism @googlecloud @kubernetesio @jfokus Configuration Map
64. @saturnism @googlecloud @kubernetesio @jfokus Secrets
65. @saturnism @googlecloud @kubernetesio @jfokus Namespace
66. @saturnism @googlecloud @kubernetesio @jfokus Service discovery Read service IP addresses via environmental variables
67. @saturnism @googlecloud @kubernetesio @jfokus Service discovery Kubernetes API or… DNS Lookups! ping redis
68. @saturnism @googlecloud @kubernetesio @jfokus Before Stateful Set - Stateless Cattles Deployment Replicas → 1 Pod frontend Pod - cb-axk3u type = Couchbase version = 1.0 Pod CB-Disk Volume Mount
69. @saturnism @googlecloud @kubernetesio @jfokus Before Pet Set - Stateless Cattles Deployment Replicas → 2 Pod frontend Pod - cb-axk3u type = Couchbase version = 1.0 Pod CB-Disk Pod frontend Pod - cb-a94kd type = Couchbase version = 1.0 Volume Mount
70. @saturnism @googlecloud @kubernetesio @jfokus Stateful Set Stateful Set Replicas → 1 Pod frontend Pod - cb-0 type = Couchbase version = 1.0 cb-0 Volume Mount Automatic Provisioning
71. @saturnism @googlecloud @kubernetesio @jfokus Stateful Set Stateful Set Replicas → 2 Pod frontend Pod - cb-0 type = Couchbase version = 1.0 Pod Pod frontend Pod - cb-1 type = Couchbase version = 1.0 cb-0 Volume Mount cb-1 Volume Mount
72. @saturnism @googlecloud @kubernetesio @jfokus Wait a second, how about the disks? volumeClaimTemplates: - metadata: name: www annotations: volume.alpha.kubernetes.io/storage-class: anything spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi
73. @saturnism @googlecloud @kubernetesio @jfokus Stateful Set Stateful Set Replicas → 2 Pod frontend Pod - cb-0 type = Couchbase version = 1.0 Pod Pod frontend Pod - cb-1 type = Couchbase version = 1.0 cb-0 Volume Mount Automatic Provisioning cb-1 Volume Mount
74. @saturnism @googlecloud @kubernetesio @jfokus Next Step? Data Centers as One!
75. @saturnism @googlecloud @kubernetesio @jfokus UI CLI API Control Plane Servers Admin containers containers containers containers containers containers containers containers containers containers containers containers containers containers containers Cluster / Data Center / Availability Zone
76. @saturnism @googlecloud @kubernetesio @jfokus UI CLI API API Kubernetes on Kubernetes on Kubernetes on Premise Federation
77. @saturnism @googlecloud @kubernetesio @jfokus Higher Availability Easy Application Migration Avoid Vendor Lock-in Capacity Overflow Cross-cluster Load Balancer Your users Cluster 1 Cluster 2 Cluster 3 Use Cases
78. @saturnism @googlecloud @kubernetesio @jfokus Provider 1 Zone A Zone B Provider 2 Zone C Provider 1 Zone D Challenges
79. @saturnism @googlecloud @kubernetesio @jfokus Geographically aware DNS gets clients to the "closest" healthy cluster. Standard Kubernetes service load balancing within each cluster. Can be extended to divert traffic away from "healthy-but-saturated" clusters. Cross-cluster Load Balancing
80. @saturnism @googlecloud @kubernetesio @jfokus Location affinity Strictly coupled pods/applications • High bandwidth requirements • Low latency requirements • High fidelity requirements • Cannot easily span clusters Loosely coupled • Opposite of above • Relatively easily distributed across clusters Preferentially coupled • Strongly coupled but can be migrated piecemeal.
81. @saturnism @googlecloud @kubernetesio @jfokus Location affinity continued... Negative Affinity • Don't run my replicas in the same failure domain (host/rack/zone) Topology • Same host • Same rack • Same zone • Same metro region • Same sub-continent Absolute affinity
82. @saturnism @googlecloud @kubernetesio @jfokus What can we federate? As of Kubernetes 1.5 Namespace Deployment Replica Set Secret ConfigMap Service Ingress
83. @saturnism @googlecloud @kubernetesio @jfokus Federation Control Plane Cluster 2 us-central1-b Cluster 1 us-east1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b API API API API Cluster 2 - us-central1-b Federation API Server Federation Controller Federation Key/value store (etcd) Federation API contexts: - context: cluster: federation-cluster user: federation-cluster
84. @saturnism @googlecloud @kubernetesio @jfokus Adding a Cluster Kubernetes Cluster (Asia) Federation Control Plane kubectl --context=federation-cluster create -f clusters/gcp-asia-east1.yaml apiVersion: federation/v1beta1 kind: Cluster metadata: name: gce-asia-east1 spec: serverAddressByClientCIDRs: - clientCIDR: "0.0.0.0/0" serverAddress: "https://..." secretRef: name: gce-asia-east1 API kubeconfig
85. @saturnism @googlecloud @kubernetesio @jfokus Cluster Federation Kubernetes Cluster 1 (Google Cloud) Kubernetes Cluster 2 (On-Prem) Kubernetes Cluster 3 (Another Cloud) Federation Control Plane kubectl
86. @saturnism @googlecloud @kubernetesio @jfokus Cluster Federation Kubernetes Cluster 1 (Google Cloud) Kubernetes Cluster 2 (On-Prem) Kubernetes Cluster 3 (Another Cloud) Federation Control Plane kubectl create -f nginx-service.yaml nginx Service nginx Service nginx Service
87. @saturnism @googlecloud @kubernetesio @jfokus Cluster Federation Kubernetes Cluster 1 (Google Cloud) Kubernetes Cluster 2 (On-Prem) Kubernetes Cluster 3 (Another Cloud) Federation Control Plane kubectl create -f nginx-service.yaml nginx Service nginx Service nginx Service DNS
88. @saturnism @googlecloud @kubernetesio @jfokus Kubernetes Cluster 1 (Google Cloud) Kubernetes Cluster 2 (On-Prem) Kubernetes Cluster 3 (Another Cloud) svc app Clusters 1 and 3 DNS nslookup app.default.federation.svc.federation.com svc app
89. @saturnism @googlecloud @kubernetesio @jfokus Ecosystem
90. @saturnism @googlecloud @kubernetesio @jfokus minikube
91. @saturnism @googlecloud @kubernetesio @jfokus kompose
92. @saturnism @googlecloud @kubernetesio @jfokus kubeadm
93. @saturnism @googlecloud @kubernetesio @jfokus kops
94. @saturnism @googlecloud @kubernetesio @jfokus helm
95. @saturnism @googlecloud @kubernetesio @jfokus kubefed
96. @saturnism @googlecloud @kubernetesio @jfokus Resources Spring Boot: https://github.com/saturnism/spring-boot-docker Federation: https://github.com/saturnism/buttonmasher Visualizer: https://github.com/saturnism/gcp-live-k8s-visualizer Hands-on Lab: http://bit.ly/k8s-lab Raspberry Pi Cluster: http://bit.ly/k8s-rpi Google Container Engine: https://cloud.google.com/container-engine/
97. @saturnism @googlecloud @kubernetesio @jfokus Thanks! Images by Connie Zhou http://kubernetes.io http://bit.ly/1QLg5E1