The Container End-Game: An Introduction To Kubernetes And Minikube

More and more people either already have adopted Docker containers for their dev work, or are actively looking into it. Not many of them are using containers in production yet, and are unsure how to go about doing so. After years of uncertainty, the industry is moving towards a common platform for doing just that.

In this talk, Stuart will introduce you to Kubernetes - the de-facto standard approach to running containers in production. He’ll show you how to map the things you know from traditional infrastructure and Docker onto Kubernetes. He’ll also introduce you to minikube - a way to run (most of) Kubernetes on a local machine. He’ll finish by covering some of the areas where Kubernetes needs de-Googling in the future.

Presented at PHP Hampshire on 14th November, 2018.

Transcript

1. A presentation by @stuherbert
 for @GanbaroDigital The Container End-Game An Introduction To 
 Kubernetes And Minikube
2. Industry veteran: architect, engineer, leader, manager, mentor F/OSS contributor since 1994 Talking and writing about PHP since 2004 Chief Software Archaeologist Building Quality @GanbaroDigital About Stuart
3. Follow me I do tweet a lot about non-tech stuff though :) @stuherbert
4. @GanbaroDigital ?? ?? Do you use Docker containers for dev work?
5. @GanbaroDigital https://speakerdeck.com/stuartherbert/ docker-for-php-dev-environments
6. @GanbaroDigital ?? ?? How about in Production?
7. @GanbaroDigital
8. @GanbaroDigital Kubernetes is the end-game for Docker containers
9. @GanbaroDigital
10. @GanbaroDigital Minikube gives you (most of) Kubernetes on your dev box*
11. @GanbaroDigital
12. @GanbaroDigital CNCF is helping Kubernetes become the de-facto platform
13. @GanbaroDigital Kubernetes is a paradigm shift
14. @GanbaroDigital It's impossible to talk about Kubernetes without talking about Google.
15. @GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube 3. Thinking In Kubernetes 4. De-Google-ing Kubernetes
16. @GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube 3. Thinking In Kubernetes 4. De-Google-ing Kubernetes
17. @GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube 3. Thinking In Kubernetes 4. De-Google-ing Kubernetes
18. @GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube 3. Thinking In Kubernetes 4. De-Google-ing Kubernetes
19. @GanbaroDigital This is my experience to date. I'm here to learn from you too.
20. @GanbaroDigital Introducing Kubernetes
21. @GanbaroDigital “ Kubernetes is a DIY platform for managing and running containers in production.
22. @GanbaroDigital That doesn't tell you what it is, only what you can use it for.
23. @GanbaroDigital I find it helpful to understand K8S by looking under the hood.
24. @GanbaroDigital Kubernetes Constituents
25. @GanbaroDigital VM
26. @GanbaroDigital VM VM VM VM VM VM
27. @GanbaroDigital VM VM VM VM VM VM
28. @GanbaroDigital VM VM VM VM VM VM Masters
29. @GanbaroDigital VM VM VM VM VM VM Masters Nodes
30. @GanbaroDigital VM VM VM VM VM VM data Masters Nodes
31. @GanbaroDigital VM VM VM VM VM VM data API Masters Nodes
32. @GanbaroDigital VM VM VM VM VM VM data API Scheduler Masters Nodes
33. @GanbaroDigital So far, this is very similar to older infrastructure solutions.
34. @GanbaroDigital What comes next is what makes Kubernetes something new.
35. @GanbaroDigital VM VM VM VM VM VM data API Scheduler Masters Nodes
36. @GanbaroDigital VM VM VM VM VM VM data API Scheduler internal controllers Masters Nodes
37. @GanbaroDigital VM VM VM VM VM VM data API Scheduler internal controllers cloud controllers Masters Nodes
38. @GanbaroDigital Pod Pod Pod Pod CNI
39. @GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS EBS EBS EBS EBS
40. @GanbaroDigital Pod Pod Pod Pod CNI LB LB EBS EBS EBS EBS EBS EBS EBS EBS
41. @GanbaroDigital VM VM VM VM VM VM data API Scheduler internal controllers cloud controllers Masters Nodes
42. @GanbaroDigital VM VM VM VM VM VM data API Scheduler internal controllers cloud controllers Masters Nodes Control plane
43. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd Masters Nodes Control plane internal controllers cloud controllers
44. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox Masters Nodes Control plane internal controllers cloud controllers
45. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API Masters Nodes Control plane internal controllers cloud controllers
46. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking Masters Nodes Control plane internal controllers cloud controllers
47. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy Masters Nodes Control plane internal controllers cloud controllers
48. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers cloud controllers
49. @GanbaroDigital
50. @GanbaroDigital https://kubernetes.io
51. @GanbaroDigital Introducing Minikube
52. @GanbaroDigital Minikube takes most of Kubernetes and sticks it in a single VM on your local machine
53. @GanbaroDigital Scaling Kubernetes Down
54. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers cloud controllers
55. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers
56. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers
57. @GanbaroDigital VM VM VM data API Scheduler containerd worker API emulated networking proxy Nodes Control plane Workload internal controllers
58. @GanbaroDigital data API Scheduler containerd worker API emulated networking proxy Control plane Workload internal controllers
59. @GanbaroDigital VM data API Scheduler containerd worker API emulated networking proxy Control plane Workload Boot2Docker internal controllers
60. @GanbaroDigital
61. @GanbaroDigital https://github.com/kubernetes/minikube
62. @GanbaroDigital How Minikube Is Different
63. @GanbaroDigital ?? ?? The things that are missing ... how important are they?
64. @GanbaroDigital We lose the cloud controllers. Because Minikube isn't running inside a cloud environment.
65. @GanbaroDigital That takes away networking and storage that you'll definitely use on full-fat Kubernetes.
66. @GanbaroDigital We lose any networking that relies on load balancers. This is mostly network access into containers on Kubernetes.
67. @GanbaroDigital Pod Pod Pod Pod CNI LB LB EBS EBS EBS EBS EBS EBS EBS EBS
68. @GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS EBS EBS EBS EBS
69. @GanbaroDigital There is a project called MetalLB that can fill the gap. It isn't integrated into Minikube at this time.
70. @GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS EBS EBS EBS EBS
71. @GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS EBS EBS EBS EBS EBS EBS EBS
72. @GanbaroDigital Ingress Controller • Works on Minikube • Works on K8S in the cloud • Supports HTTP/HTTPS only • HTTPS is terminated at Ingress
73. @GanbaroDigital We lose attached storage. This affects every container that we attach extra volumes to.
74. @GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS EBS EBS EBS EBS EBS EBS EBS
75. @GanbaroDigital Pod Pod Pod Pod CNI Ingress
76. @GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS HostPath EBS HostPath EBS HostPath EBS HostPath
77. @GanbaroDigital HostPath Volumes • VM folders mounted into containers • When the VM is deleted, all data is lost • Create volumes under /data to keep data between VM reboots
78. @GanbaroDigital Because Minikube runs in a VM, mounting volumes from the host box isn't always an option.
79. @GanbaroDigital And it isn't always reliable :(
80. @GanbaroDigital We also lose the sandbox. Containers may* behave differently as a result. * only if your K8S uses a sandbox
81. @GanbaroDigital Why is all this important to know upfront?
82. @GanbaroDigital You can't use identical deployments on both Minikube and full-fat Kubernetes.
83. @GanbaroDigital Living With Minikube
84. @GanbaroDigital Minikube does its best, but Kubernetes just doesn't scale down nicely today.
85. @GanbaroDigital Minikube is a bit of a RAM and CPU hog.
86. @GanbaroDigital This is a mix of VM overhead and K8S components that burn CPU even when idle.
87. @GanbaroDigital I've had to upgrade my dev box to use Minikube. So has the customer I am working with.
88. @GanbaroDigital
89. @GanbaroDigital
90. @GanbaroDigital Intel NUCs are my secret weapon! 32GB of RAM, NVMe storage, and you can reinstall everything when you screw up.
91. @GanbaroDigital When Minikube Is Useful
92. @GanbaroDigital ?? ?? What is Minikube good for?
93. @GanbaroDigital Use Minikube for faster iteration of your K8S objects.
94. @GanbaroDigital Use Minikube on something like an Intel NUC as a mini K8S server.
95. @GanbaroDigital Stick with Docker Compose for local dev work.
96. @GanbaroDigital If you must host dev work on Kubernetes, spin up a K8S dev cluster on a cloud provider.
97. @GanbaroDigital Thinking in Kubernetes
98. @GanbaroDigital “ Kubernetes is a paradigm shift.
99. @GanbaroDigital Deploying Onto Kubernetes
100. @GanbaroDigital apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80 Kubernetes objects
101. @GanbaroDigital Kubernetes Objects • YAML descriptions • Define what we want on K8S • Uploaded via kubectl CLI tool
102. @GanbaroDigital VM VM VM data API Scheduler Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80 internal controllers cloud controllers
103. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80
104. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80
105. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80
106. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80
107. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: ReplicaSet metadata: name: nginx-deployment-ykzjud ...
108. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: ReplicaSet metadata: name: nginx-deployment-ykzjud ...
109. @GanbaroDigital This cycle continues until 'Pods' have been created / updated / deleted in the data store.
110. @GanbaroDigital A Pod is the unit of deployment.
111. @GanbaroDigital A Pod contains 1 (or more) containers.
112. @GanbaroDigital IP addresses are assigned to Pods, not containers.
113. @GanbaroDigital Containers inside the same Pod can talk to each over via localhost:<port-number>
114. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload
115. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload
116. @GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload
117. @GanbaroDigital We don't tell K8S "start this container". We tell K8S "we want these containers running" and K8S makes it happen.
118. @GanbaroDigital ... and keeps it running no matter what until we change our mind.
119. @GanbaroDigital Deployments • Objects describe desired state • K8S updates active state to match • K8S restores active state when things go wrong
120. @GanbaroDigital Deployments • Objects describe desired state • K8S updates active state to match • K8S restores active state when things go wrong
121. @GanbaroDigital Deployments • Objects describe desired state • K8S updates active state to match • K8S restores active state when things go wrong
122. @GanbaroDigital The "desired state ⬌ active state" approach is Inversion of Instruction.
123. @GanbaroDigital The result? Invisible Infrastructure (as far as devs are concerned)
124. @GanbaroDigital Containers Aren't VMs
125. @GanbaroDigital Running on Kubernetes is nothing like running VMs on AWS et al.
126. @GanbaroDigital Running on Kubernetes is nothing like running containers locally.
127. @GanbaroDigital ... but it looks close enough to make this hard to grok.
128. @GanbaroDigital “ Old habits die hard.
129. @GanbaroDigital Let's look at an important example: Backups
130. @GanbaroDigital java /var/lib/myapp Pod
131. @GanbaroDigital java /var/lib/myapp VM tar SSH
132. @GanbaroDigital java /var/lib/myapp Pod tar SSH
133. @GanbaroDigital java /var/lib/myapp Pod tar SSH ✗
134. @GanbaroDigital java /var/lib/myapp Pod tar SSH ✗ ✗
135. @GanbaroDigital Containers are black boxes.
136. @GanbaroDigital You don't SSH into them to take backups.
137. @GanbaroDigital You don't SSH into them to patch them.
138. @GanbaroDigital You don't SSH into them ever.
139. @GanbaroDigital java /var/lib/myapp Container tar Host
140. @GanbaroDigital java /var/lib/myapp Pod tar Host
141. @GanbaroDigital java /var/lib/myapp tar Host ✗ Pod
142. @GanbaroDigital java /var/lib/myapp tar Host ✗ ✗ Pod
143. @GanbaroDigital Only running containers have access to their filesystems.
144. @GanbaroDigital java /var/lib/myapp Pod
145. @GanbaroDigital java /var/lib/myapp VM stop
146. @GanbaroDigital java /var/lib/myapp VM ✗ stop
147. @GanbaroDigital You can't stop the container's main process to get consistent filesystem backups.
148. @GanbaroDigital java /var/lib/myapp Pod ✗ stop
149. @GanbaroDigital java /var/lib/myapp Pod ✗ stop
150. @GanbaroDigital java /var/lib/myapp Pod 2
151. @GanbaroDigital Kubernetes knows what we want the cluster to look like: "desired state"
152. @GanbaroDigital When "active state" deviates from "desired state", Kubernetes attempts to put things back.
153. @GanbaroDigital The (portable) solution? Use K8S rollouts to replace the app container with a specialist backup container.
154. @GanbaroDigital "I have a container for that" - the K8S Way
155. @GanbaroDigital Every single practice you already have from traditional infrastructure has to be reinvented.
156. @GanbaroDigital “ Running Kubernetes, you are Alice through the looking glass.
157. @GanbaroDigital Once you're used to the Kubernetes approach, going back to traditional infrastructure just feels wrong.
158. @GanbaroDigital De-Googling Kubernetes
159. @GanbaroDigital We can all adopt Kubernetes thanks to Google.
160. @GanbaroDigital There is still work to do to adapt Kubernetes to the world beyond Google.
161. @GanbaroDigital The Google Bubble
162. @GanbaroDigital VM Minkube
163. @GanbaroDigital VM VM VM VM VM VM Masters Nodes
164. @GanbaroDigital
165. @GanbaroDigital
166. @GanbaroDigital “ Problems change with volume and scale.
167. @GanbaroDigital Google is solving problems you don't have.
168. @GanbaroDigital Google's solutions sometimes don't scale down.
169. @GanbaroDigital A surprising amount of Googlers seem to lack normal company experience.
170. @GanbaroDigital Or maybe Google just has blind spots like anyone else?
171. @GanbaroDigital Most third-party containers don't work out-of-the-box on Kubernetes.
172. @GanbaroDigital
173. @GanbaroDigital Through the CNCF, Kubernetes is evolving to suit a general audience.
174. @GanbaroDigital You absolutely CAN adopt Kubernetes today.
175. @GanbaroDigital You have to adapt if you want to adopt Kubernetes.
176. @GanbaroDigital
177. Thank You How Can We Help You? A presentation by @stuherbert
 for @GanbaroDigital