Creating a fast Kubernetes Development Workflow

Transcript

1. @BastianHofmann Creating a fast Kubernetes Development Workflow Bastian Hofmann
3. Container orchestration platform
4. Deploy, run and scale your services in isolated containers
5. Very Powerful
6. Large community
7. Lot’s of large company backers
8. No vendor lock in
9. Standardized APIs
10. Runs on
11. Your laptop
13. Bare metal
14. Cloud Providers
15. AWS
16. Azure
17. Google Cloud Platform
18. And if you don't want to install and maintain Kubernetes yourself
19. Managed Kubernetes
21. Easy setup
22. Easy upgrades
23. Easy scaling
24. Features
25. Load Balancing
26. Distributed Persistent Storage
27. Backups
28. Monitoring
29. Support
30. You can focus on what is important
31. But this talk is about how to use Kubernetes
32. Not only for production workloads
33. But in your development workflows
34. Kubernetes has standardized apis
35. More and more integrations
36. Great tools
37. Agenda
38. Introduction to Kubernetes
39. Deployment of a simple application
40. Deployment of a micro-service application
41. Some tools for development with Kubernetes
42. But first
43. Why containers?
44. Services run in isolation
45. Everything needed to run a service in one image
46. Make things …
47. Easier to develop
48. Easier to deploy
49. Easier to upgrade system dependencies
50. Easier to scale
51. Better resource usage
52. #safeThePlanet
53. Kubernetes helps you to deploy, run and scale containers
54. Let’s define some core concepts and terminology first
55. Kubernetes Cluster
56. • A docker image built from a Dockerfile that contains everything a service needs to run Image
57. • A container runs a docker image. • Only 1 process can run inside of a container Container
58. • A group of 1 or more containers • Same port space • Within a Pod: communication over localhost • Every Pod has it's own IP • All Pods can talk with each other • IPs change all the time Pod
59. • Defines and manages how many instances of a pod should run • ReplicaSet is tied to a specific definition of a Pod which is tied to specific image versions of the container • Image versions in ReplicaSets can't be updated Replica Set
60. • Manages updates and rollbacks of replica sets Deployment
61. • Internal LoadBalancer • Makes all pods matching a set of labels accessible through a stable, internal IP address • You can attach external IP address through an cloud LoadBalancer Service
62. • Makes a service accessible to the outside of Kubernetes through an ingress controller (e.g. nginx) • Traffic is routed by routing rules, usually Host header Ingress
63. • A physical server • Containers get distributed automatically Node
64. • Key/Value storage for configuration ConfigMap
65. • Key/Value storage for configuration, usually passwords. Secret
66. • Volumes can be mounted into a container to access a ConfigMap, Secret, persistent volumes with network storage or a folder on the node Volumes
67. • Dedicated environment to deploy services in Namespaces
68. CronJobs, DaemonSets, StatefulSets, ...
69. Everything is a resource
70. You interact with Kubernetes by creating, receiving, updating and deleting resources
71. Kubernetes has controllers to listen on these interactions and get the cluster in the desired state
72. The Kubernetes API can be extended with additional Resources and Controllers
73. CustomResourceDefinitions
74. Certificate, Backup, Restore, MySQLCluster, Function, ...
75. kind: Deployment apiVersion: extensions/v1beta1 metadata: name: symfony-demo spec: template: spec: containers: - name: symfony-demo image: symfony-demo:1.1.0 ports: - containerPort: 80
76. $ kubectl apply -f deployment.yaml
77. $ kubectl get deployments NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE symfony-demo 1 1 1 1 21h
78. $ kubectl get deployment symfony-demo -o yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: annotations: ... spec: ... template: ... spec: containers: - name: symfony-demo image: symfony-demo:1.1.0
79. $ kubectl delete deployment symfony-demo
80. Practical example
81. We need a cluster
82. Let’s deploy an application
83. DEMO
84. What did just happen?
86. Deployment created
87. Sees new Deployment And creates new ReplicaSet with 1 desired replica
88. Sees new ReplicaSet and Creates Pod for ReplicaSet
89. Sees new unscheduled Pod and Schedules it to Node
90. Sees it is supposed to start a Pod And starts its Containers
91. Service created
92. Sees the new Service And configures IP Table Rules and DNS entries
93. Sees the new Service has the Type LoadBalancer and creates An External LB at the Cloud Provider
94. What about Configuration
95. DEMO
96. What about TLS and DNS
97. You don't want to implement TLS certificate handling in every public service
98. Ingress Controller and cert-manager
99. The ingress controller (nginx) listens on Ingress Resources and configures itself to route incoming traffic based on the host header to the correct running pods
100. Cert-manager listens on Ingresses and if they want TLS, requests a certificate from LetsEncrypt
101. External-DNS listens on Ingresses and creates DNS entries at DigitalOcean
102. How is traffic routed to the Pod
103. OpenStack LoadBalancer
104. DEMO
105. What about Persistent Storage and Databases
106. DEMO
107. Writing this YAML files is tedious
108. YAML files are tied to a specific version and a specific environment
109. Production
110. Staging
111. Development
112. Per Development team
113. Per branch
114. Per developer
115. Built-in
116. Namespaces
117. Still we'd need to maintain multiple very similar YAML files with slightly different versions and configuration.
118. "Templating"
119. Great tools because of standardized Kubernetes API
120. Helm
122. Allows to install applications
123. So called "charts"
124. Writing your own charts if fairly easy
125. Charts can depend on other charts
126. Multiple deployments of one chart possible
127. Different namespaces
128. Different release names
129. Configuration over values
131. Different versions
132. Different ingress urls
133. $ helm install stable/wordpress --namespace bastian --name my-wordpress --values dev.yaml --values bastian.yaml
134. Still:
135. Make a code change
136. Build docker image
137. Push docker image
138. Run helm install/upgrade with new image version
139. Can this be quicker?
140. Tilt
141. Watches for changes
142. Rebuilds docker image
143. Deploys to Kubernetes
144. You can use your helm templates
145. $ tilt up
146. Demo application
147. web quote-svc hello-svc
148. Not all services have an ingress
149. Accessing Kubernetes from the outside
150. web quote-svc hello-svc
151. Getting a shell in a running container
152. $ kubectl exec $POD_NAME -i -t -- /bin/bash
153. Port forwarding through kubectl
154. $ kubectl port-forward pod/$POD_NAME 8080:80
155. $ kubectl port-forward service/$SERVICE_NAME 8080:80
156. What about step debugging?
157. Of course you can run everything locally
158. But you develop only on one service
159. There may be lots of services
160. You don't want to expose all services publicly
161. Port-forwarding all services is also work
162. Telepresence
164. Creates a two-way proxy between the Kubernetes cluster and you
165. $ telepresence T: Starting proxy with method 'vpn-tcp'... @fhgbvx65xg|bash-3.2$ curl http://quote-svc/quote | jq '.' [ { "ID": 503, "title": "stefan sagmeister", "content": "<p>...</p>\n", "link": "https://quotesondesign.com/stefan- sagmeister-2/" } ]
166. Swap a running deployment in the cluster with a local process
167. ... or a locally running docker container
168. $ telepresence --swap-deployment quote-svc --namespace dev-flow-demo --expose 3000 --run npm run debug T: Starting proxy with method 'vpn-tcp',... T: Forwarding remote port 3000 to local port 3000.... > quote-svc@1.0.0 debug /Users/bhofmann/forge_test/quote- svc > nodemon --inspect quote-svc.js [nodemon] watching: *.* [nodemon] starting `node --inspect quote-svc.js` Debugger listening on ws://127.0.0.1:9229/83aa27ac- d879-4b50-a228-440354cca791 quote svc listening on port 3000!
169. Demo
170. Summary
171. Powerful
172. Helpful
173. Great tooling because of common APIs
174. Especially great if you have multiple services and don't want to run everything locally
175. I just picked helm, tilt and telepresence. There is more for different use-cases.
176. http:/ /speakerdeck.com/ u/bastianhofmann
177. https:/ /github.com/bashofmann/ kubernetes-dev-flow-demo
178. mail@bastianhofmann.de https:/ /twitter.com/BastianHofmann