GitOps - is this something for me?

https://www.continuouslifecycle.de/veranstaltung-15122-0-gitops--ist-das-was-fuer-mich.html

Transcript

1. // GITOPS - IS THIS SOMETHING FOR ME? Johannes Schnatterer, Cloudogu GmbH Version: 202211160730-bd22f6c @schnatterer@floss.social @jschnatterer 2
2. Agenda • Basics • Tools • Challenges • Demo 3
3. GitOps basics 5
4. Origin: blog post by Weaveworks, August 2017 Use developer tooling to drive operations weave.works/blog/gitops-operations-by-pull-request 6
5. "Classic" Continuous Delivery ("CIOps") Developer Git Repo CI Server Continuous Integration + Continuous Delivery K8s Cluster push pull deploy imperative, once GitOps K8s Cluster Developer Git Repo CI Server Continuous Integration Continuous Delivery GitOps operator push pull pull deploy declarative, continuously 7
6. GitOps Principles The desired state of a GitOps managed system must be: 1 Declarative 2 Versioned and Immutable 3 Pulled Automatically 4 Continuously Reconciled github.com/open-gitops/documents/blob/main/PRINCIPLES.md 8
7. GitOps vs DevOps • DevOps is about collaboration of formerly separate groups (mindset) • GitOps focuses on ops (operating model) • GitOps could be used with or without DevOps and vice versa • Still, GitOps might be... The right way to do DevOps Alexis Richardson 9
8. K8s Cluster Developer Git Repo CI Server GitOps operator push pull pull deploy Advantages of GitOps • No access to cluster from outside (might also solve firewall/zone issues) • No credentials on CI server (neither cluster access nor for apps) • Forces declarative description • IaC is auditable • Self-healing • Scalability - one repo many applications 10
9. GitOps tools 12
10. GitOps tool categories • GitOps operators/controllers • Supplementary GitOps tools • (Tools for operating cloud infra) 13
11. GitOps operators/controllers 14
12. Supplementary GitOps tools • Secrets • KMS, e.g. ... • + K8s Integration • Operator • Container Storage Interface (CSI) driver • Side car (injector) • Helm/Kustomize plugin • GitOps Operator: native support or plugin • Backup / restore • Deployment Strategies - Progressive Delivery • ... 15
13. See also • ( ) • General tool comparison, • tips on criteria for tool selection, • comparison of ArgoCD and Flux • ArgoCD vs Flux (10/2022) cloudogu.com/blog/gitops-tools iX 4/2021 community.cloudogu.com/t/flux-or-argocd-the-great-controversy 16
14. Challenges with GitOps 18
15. Downsides • More infra necessary • Steep learning curve 19
16. GitOps infra • GitOps Operator comprises several applications • Cause ops efforts: maintenance, alerts argo-cd.readthedocs.io/en/stable/assets/argocd_architecture.png fluxcd.io/img/diagrams/gitops-toolkit.png 20
17. Learning curve • New concepts and tools for developers and platform teams • Adapt deployment process • Migrate applications • Adapt error handling and alerting • avoid failing late and silently • accustom to new notification mechanism • still, reason might be difficult to pinpoint 21
18. Day two questions • How to realize local dev env? • How to delete resources? • How to structure repos and folders? • How to realize different stages/environments? • Role of CI server? • ... 22
19. Local development • Option 1: Deploy GitOps operator and Git server on local cluster complicated • Option 2: Just carry on without GitOps. 23
20. How to delete resources? • garbage collection / resource pruning disabled by default • Enable from beginning avoid manual interaction • Unfortunately, still often unreliable / too defensive (?) 24
21. Repo and folder structure • No standard for structures (intentionally) Conway's law • Repo patterns: Monorepo vs Polyrepo (per app, team, stage) • Within repo: folder/branch structure for stage, team, app • More options: • Topology: GitOps controller (s) Cluster(s) / Namespaces • GitOps controller-specific config community.cloudogu.com/t/gitops-patterns-for-repository-and-folder-directory-structure 25
22. GitOps Chasm Infra • repos • folders • branches • clusters • namespaces • controller instances • controller-specific config Mapping? Real-world • company/departments • teams • projects • applications • microservices • stages/environments • customers • tenants • etc. 26
23. App repo vs GitOps repo K8s Cluster Developer App Repo GitOps Repo pull CI Server GitOps operator OCI Registry push app code push infra code pull push pull deploy GitOps tools: Put infra in separate repo! See argo-cd.readthedocs.io/en/release-2.5/user-guide/best_practices 27
24. Disadvantages • Separated maintenance & versioning of app and infra code • Review spans across multiple repos • Local dev more difficult • Static code analysis for IaC code not possible How to avoid those? 28
25. Using CI-Server with GitOps part 1 K8s Cluster Developer App Repo GitOps Repo CI Server GitOps operator OCI Registry push app +infra code pull push infra code push pull pull deploy 29
26. Advantages • Single repo for development: higher efficiency • Shift left: static code analysis + policy check on CI server, e.g. yamlint, kubeval, helm lint, conftest • Automated staging (e.g. PR creation, namespaces) • Simplify review by adding info to PRs 30
27. Disadvantages • Complexity in CI pipelines Recommendation: Use a plugin or library, e.g. cloudogu/gitops-build-lib • Redundant code 31
28. Alternative: Refer to app repo K8s Cluster Developer App Repo GitOps Repo CI Server GitOps operator OCI Registry push app +infra code refer to pull push pull pull pull deploy e.g. fluxcd.io/flux/guides/repository-structure 32
29. Stage promotion • "GitOps - Operations by Pull Request" • Repo structure: Use folders not branches • But: create short-lived branches and PR • Merge is promotion/deployment 33
30. Implementing stage promotion Who bumps versions in GitOps repo, creates branch and PR? • Manual: Human pushes branch and create PR • Image Updater: Operator pushes branch, create PR manually • CI Server: Build job pushes branch, creates PR • Renovate Bot: Bot pushes branch, creates PR 34
31. Image updater K8s Cluster Developer Git Repo CI Server GitOps operator OCI Registry push pull push pull + watch pull + push deploy GitOps operator can update image version in Git • • github.com/argoproj-labs/argocd-image-updater fluxcd.io/docs/guides/image-update 35
32. Using CI-Server with GitOps part 2 K8s Cluster Developer App Repo GitOps Repo CI Server GitOps operator OCI Registry push app +infra code pull push infra code +create PR push pull pull deploy 36
33. Stage promotion using renovate bot K8s Cluster GitOps operator Developer Git Repo Renovate bot create PR watch CI Server OCI Registry push pull push pull pull deploy github.com/renovatebot/renovate 37
34. As example: Our approach • Repo pattern: Trunk-based repo per team, folder per stage+app (mixed with repo per app) • IaC either • lives in app repo and is pushed by CI-Server or • in GitOps repo (3rd party apps). • Promotion between stages: • commit to staging folder only ( protect production), • create short lived branches and pull requests for prod ├── staging │ └── application │ └── deployment.yaml └── production └── application └── deployment.yaml 38
35. Demo Your Host K3d Container SCM-Manager Docker Daemon Registry Jenkins ArgoCD Staging (6.) + Production (7.) App Repos GitOps Repos run 1. push 7. review 8. accept PR 2. pull 4. push IaC + Create PR 3. push image 5. pull deploy cloudogu/gitops-playground 40
36. BTW: More Features to explore Your Host K3d Container SCM-Manager Docker Daemon API-Server Grafana Prometheus External Secrets Operator Vault Mailhog Registry Jenkins GitOps Operators App Repos GitOps Repos run view metrics create secret read mail push accept PR pull push IaC + Create PR push image pull read secret deploy create secret send alert send query Scrape metrics 41
37. Is GitOps something for me? Has advantages, once established Don't underestimate learning curve and migrations 43
38. Johannes Schnatterer, Cloudogu GmbH • • • • Join my team: cloudogu.com/gitops GitOps Resources Community Trainings Consulting cloudogu.com/join/cloud-engineer @schnatterer@floss.social @jschnatterer 44
39. Image sources • Basics: • Tools: • Challenges: • Is GitOps something for me? https://pixabay.com/illustrations/question-mark-important-sign- 1872665/ https://pixabay.com/photos/tools-knives-wrenches-drills-1845426/ https://unsplash.com/photos/bJhT_8nbUA0 https://pixabay.com/illustrations/question-mark-question-response- 1020165/ 45