Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWS MENA Community Day - GitOps on AWS_ Codifyi...

AWS MENA Community Day - GitOps on AWS_ Codifying multi-cloud operations

AWS MENA Community Day - GitOps on AWS_ Codifying multi-cloud operations By Mahmoud Saada
@saadazzz

AWS MENA Community

September 26, 2020
Tweet

More Decks by AWS MENA Community

Other Decks in Technology

Transcript

  1. • Roles: SE, TL, SRE, CRE • Startups and Enterprises

    • Industries: HR, AI, Fin-tech, Infrastructure • Companies: TargetCW, ADP, Agolo, Weaveworks • Meetups: Docker NYC, Uber Tracing NYC • Open Source: grafana, prometheus, terraform, helm, helm operator, eksctl, elasticsearch, … • Certified Kubernetes Administrator Background
  2. GitOps is the practice of using Git to declaratively define

    desired state and Continuous Delivery agents (Flux) to automate the reconciliation of current state to desired/intended state; effectively decoupling CI and CD GitOps Definition
  3. Weaveworks’s Gitops Tools • Flux is an async-pull continuous delivery

    agent that runs as a controller inside its target Kubernetes cluster. Flux pulls git changes and converges them with a Kubernetes state. It was created at Weaveworks. • Helm Operator is a Kubernetes operator that watches for a CRD called HelmRelease and renders it into k8s resources
  4. Weaveworks: The GitOps Company Weaveworks created the GitOps methodology and

    tooling to solve our own Kubernetes management, scalability, and reliability requirements Weaveworks is a key partner with all the major infrastructure and Kubernetes vendors Weaveworks is deeply committed to the Open Source Community Weaveworks is backed by solid investors
  5. 1 The entire system is described declaratively 2 The canonical

    desired system state is versioned (with Git) 3 Approved changes to the desired state are automatically applied to the system 4 Software agents ensure correctness and alert on divergence GitOps Principles 10
  6. GitOps – An Operating Model for Cloud Native Deployment (clusters,

    apps) Monitoring Logging (Observability) Management (operations) Build GIT Test IDE Unifying Deployment, Monitoring and Management. Git as the single source of truth of a system’s desired state ALL intended operations are committed by pull request ALL diffs between intended and observed state with automatic convergence ALL changes are observable, verifiable, and auditable “Immutability Firewall” Kubernetes GitOps Continuous Integration
  7. One git path/directory per namespace Namespace Strategy /namespaces ./team1 ./team2

    ./team3 Team1 namespace Team2 namespace Team3 namespace
  8. One git path/directory per environment Directory Path Strategy /env ./develop

    ./staging ./production Develop cluster staging cluster Production cluster
  9. One git path/directory per environment, one subdirectory per namespace Mixing

    Strategies together /env ./develop ././team1 ././team2 ./staging ././team1 ././team2 ./production ././team1 ././team2 Develop cluster broken down by namespace Staging cluster broken down by namespace Production cluster broken down by namespace
  10. • Increased productivity • Enhanced developer experience • Improved stability

    • Higher reliability • Consistency and standardization • Stronger security guarantees Benefits of adopting GitOps
  11. • Not designed for programmatic updates → feature branches +

    test environments • Proliferation of Git repositories → Consolidate gitops repos + GoTK • Lack of visibility → commit messages, flux logs, GoTK grafana dashboard • Doesn’t solve centralized secret management → Sealed Secrets, Hashicorp Vault, AWS SecretsManager • Auditing isn’t as great as it sounds like → commit messages, flux logs, GoTK grafana dashboard • Lack of input validation → CI linting Common Concerns
  12. Current State Watch Desired State Reconcile Controller Current State (etcd)

    Watch Desired State (etcd) Reconcile K8S Current State (git) Watch Desired State (etcd) Reconcile GitOps Controllers Everywhere
  13. ➔ Kubernetes application platform ➔ Management of cluster and applications

    ➔ Builds on GitOps and adds enterprise features ➔ Define clusters and components using a model based system ➔ Deploy new clusters using those definitions: multiple back-ends ➔ Alerting and operations built-in Weave Kubernetes Platform
  14. Let’s see it in action EC2 (CAPI Managed) EC2 (CAPI

    Managed) Kind (on-prem Unmanaged) EKS (management cluster)
  15. Future • Gitops Toolkit ◦ FluxV2, Helm Controller, Kustomize Controller,

    Notifications, Prometheus metrics, Grafana dashboard, ... ◦ Major release expected later this year ◦ You can find documentation and quickstart examples here: https://toolkit.fluxcd.io ◦ Helm Controller will be the successor for Helm Operator ◦ Community-driven. If interested in getting involved, go to https://github.com/fluxcd/toolkit/discussions
  16. Thank you! Demo repo: https://github.com/saada/gitops-cluster-management/ Workshop (Module 7): https://weaveworks-gitops.awsworkshop.io/ EKS

    Control: https://eksctl.io Flux: https://docs.fluxcd.io/ Helm Operator: https://docs.fluxcd.io/projects/helm-operator/ The Art of Modern Ops (podcast): https://bit.ly/weave-podcast FluxV2 (coming soon): https://toolkit.fluxcd.io/ Mahmoud Saada @saada @saadazzz weave-community.slack.com