Tame your Microservices with Helm and Software Release Trains

The deployment of apps in Kubernetes is simplified with the usage of Helm, but when it comes to deploying microservice architecture things can go a bit more wild as we have to manage multiple applications on potentially multiple environments. We'll discuss the different patterns for storing the charts on Git, either as mono or multi repos. We'll go through the ways to simplify and keep DRY our charts, and lastly we'll go review the different deployment patterns with Helm, either together as software release trains, or individually as with CI/CD.

Transcript

2. Tame your Microservices with Helm and Software Release Trains Dimitris Kapanidis Harbur Cloud Solutions
3. About Me Founder at Harbur Cloud Solutions Member: - Docker Captains - Google Developer Experts Co-organizer: - Docker Barcelona Meetup - Kubernetes Barcelona Meetup 3
4. Harbur Cloud Solutions We are a cloud native consulting • DevOps Consulting Services • Cloud Native Solutions • Courses & Workshops About Us 4
5. About Us 5
6. Tame your Microservices > Content 6 Microservice Architecture Why GitOps? CI/CD with Helm Repository Patterns (& Software Release Trains with Helm) Development Styles Demo! (with GKE + GitHub actions)
7. Software Release Trains 7 App1 v1.0.0 App2 v1.5.0 App3 v2.0.0 App4 v1.1.4 SRT v1.0.0
8. 8 Software Release Trains SRT v1.0.0 PRO TEST DEV PRE
9. Everything is connected Git Cloud Native Containers Microservices Helm DevOps 9
10. Microservice Architecture > Monolith 10 Traffic Data Center 1 Data Center 2 Monolith Monolith Monolith Monolith
11. Microservice Architecture > Microservices 11 Traffic Cluster 2 Cluster 1 Data Center 1a Data Center 1b Data Center 2a Data Center 2b API Gateway SMS Payment API Gateway SMS 2 3
12. Infrastructure > Now with Microservices 12 Storage One per instance Multi-Instances Scaling Upgrade Graceful Shutdown Discovery Session Management Quorum Data Recovery Stateful Service MariaDB ElasticSearch Kafka Prometheus Multi-Instances Scaling Upgrade Graceful Shutdown Stateless Service API Gateway Rest API Frontend Auth Stateless Stateful Microservice Architecture > Stateful & Stateless
13. 13 Microservice Architecture > Kubernetes Overview Images Registry Charts Registry
14. 14 “Turn Right” “Take me Home” GitOps > Declarative vs Imperative Imperative Declarative
15. 15 GitOps > Infrastructure as Code Pull Requests Code GitOps Kubernetes Clusters
16. 16 CI / CD > Deploy with Helm Your CI / CD only needs three actions: • Push Images • Push Charts • Deploy
17. 17 CI / CD > Deploy with Helm We Need: • An App Image • An App Chart • An App Environment setup (values.yaml) e.g: $ helm install stable/ghost -f values.yaml
18. 18 CI / CD > App Image Creation Images Registry Developer git checkout docker build docker push Code Repository
19. 19 CI / CD > App Chart Creation Developer git checkout Charts Registry Chart Repository helm dep build helm push
20. 20 CI / CD > Env Deployment with Helm git checkout Kubernetes Cluster Images Registry Charts Registry Developer helm repo up docker pull helm install Env Repository
21. 21 CI / CD > Workflow Kubernetes Cluster Images Registry helm repo up docker pull helm install Env Repository Chart Repository Charts Registry Code Repository helm push docker push
22. 22 CI / CD > Workflow Kubernetes Cluster Images Registry helm repo up docker pull helm install Env Repository Chart Repository Charts Registry Code Repository helm push docker push
23. 23 Repository Patterns > Code Structure Project Repository Project Monolith Mono Repo Multi Repo App1 Repository App2 Repository ... App1 App2 Project Repository App3 Code Code
24. 24 App1 Repository App3 Repository App4 Repository App2 Repository Repository Patterns > Code Structure Code Code Code Code ALICE TEAM BOB TEAM
25. 25 Mono Repo Multi Repo App1 Repository App2 Repository ... App1 App2 App3 Charts Repository Chart Chart Repository Patterns > Charts Structure helm/charts e.g.
26. 26 App1 Repository Repository Patterns > Charts Structure Code Chart App2 Repository Code Chart App3 Repository Code Chart App4 Repository Chart Code
27. 27 Repository Patterns > Envs Structure Mono Repo Multi Repo Pre Environment Pro Environment ... Envs Repository dev pre pro pre pro : One helm release per namespace
28. 28 Repository Patterns > Envs Structure App1 Repository Code Chart App2 Repository Code Chart App3 Repository Code Chart App4 Repository Code Chart Pre Environment Pro Environment Tst Environment Tst Pre Pro
29. Repository Patterns > Software Release Trains Tst Environment Tst Release App1 App2 App3 App4 Pre Environment Pre Release App1 App2 App3 App4 29 Pro Environment Pro Release App1 App2 App3 App4 requirement s values.yaml values.yaml values.yaml requirements requirement s
30. Development Styles > Git Flow 30 develop feature branches release branches master Tag: 0.1 Tag: 1.0 Tag: 2.0 v1 v2 Tag: 2.1
31. Development Styles > Trunk Based 31 master feature branches release branches Tag: 1.0 Tag: 2.0 v1 v2 Tag: 2.1
32. 32 CI / CD > Workflow Kubernetes Cluster Images Registry helm repo up docker pull helm install Env Repository Chart Repository Charts Registry Code Repository helm push docker push
33. 33 Development Styles > Docker Image Versioning Each branch/tag is a potential deployment On each push, images are built and pushed to Registry. Image versioning should be immutable (if possible) • Tag name is immutable • Branch name is not immutable! (Immutability is important for declarative deployments) Two alternatives for branches deployment: • Versioning should include a suffix of CI BUILD number (e.g. “master-1”) • Configure Pods to redeploy always on helm upgrade (for development environments only): ◦ Pod Annotation: helm.sh/deploy-date: "{{ .Release.Time.Seconds }}" ◦ Pod container: imagePullPolicy: Always ◦ Configure Registry to allow redeploys of same version
34. 34 Development Styles > Versioning per Environment • Production Deployments ◦ by tags ◦ continuously by “master” • Staging Deployments ◦ by “release” branches ◦ continuously by “master” ◦ continuously by “staging” • Develop Deployments ◦ by “develop” branch ◦ by “feature” branches
35. 35 Development Styles > Environments • Production Environments ◦ by cluster (AZ, DC) ◦ by namespace (client) • Staging Environments ◦ for load testing ◦ for security testing ◦ for regression testing • Develop Environments ◦ by team ◦ by developer ◦ by feature branch
36. 36 Development Styles > Centralized vs Decentralized Centralized Decentralized vs - Least Privilege Principle - Role Based Access Control - Self Service - Cluster-wide Privileges - Promoting silos culture
37. 37 Demo Time!
38. 38 DevOps Culture Automation - CI/CD - GitOps - ChatOps - Metrics - Alerts - Dashboards Feedback - Retrospectives - Transparency - Blameless post-mortems - Evidence based experiments Shared Responsibility - Objectives - KPIs - Organization Culture Autonomous Teams - Self-service - Build & Deploy Agile - Move fast - React quickly - Accept Failure - Fail fast - Fail once
39. Thank You! Innovating Container Delivery