Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Navigating the CICD landscape in the k8s era

Andreas Evers
September 17, 2021

Navigating the CICD landscape in the k8s era

With Kubernetes, everything from deployment to monitoring to scaling is more standardized and inherently easier to automate. This presents the possibility to achieve a more effective and comprehensive Continuous Integration (CI) and Continuous Delivery (CD) experience. We can incorporate the practice of infrastructure-as-code, and take advantage of a flourishing ecosystem of tools to improve and fully automate our application deployment strategies. In this session, we’ll tour the landscape of new Kubernetes-centric tooling, including Tekton, Kustomize, and ArgoCD. We’ll also explore how GitOps can be leveraged for CI/CD.

Andreas Evers

September 17, 2021
Tweet

More Decks by Andreas Evers

Other Decks in Technology

Transcript

  1. Abstract With Kubernetes, everything from deployment to monitoring to scaling

    is more standardized and inherently easier to automate. This presents the possibility to achieve a more effective and comprehensive Continuous Integration (CI) and Continuous Delivery (CD) experience. We can incorporate the practice of infrastructure-as-code, and take advantage of a flourishing ecosystem of tools to improve and fully automate our application deployment strategies. In this session, we’ll tour the landscape of new Kubernetes-centric tooling, including Tekton, Kustomize, and ArgoCD. We’ll also explore how GitOps can be leveraged for CI/CD.
  2. KOR is a Fintech created by a global distributed team

    build up out of expert in their field. Our engineering team is centered in the EU region. Product is driven by industry expert from the US and Leadership is spread between UK, US and EU. ~ We are hiring ~ People Powered Disruption
  3. We are building Real-time insights based on regulation to make

    financial products like CDO, transparent to the market and regulators.
  4. Architecture For Scale Cloud native Event driven architecture Fully hosted

    on Amazon Web Services ~12 Petabytes per year Event Sourcing / Stream Processing Kafka as a Database
  5. package Path To Production change prod code test deploy test

    deploy promote rollback test strategic rollout reviews pull request feature branch declared state maintain test kubernetes infrastructure as code self-healing extensible api git source of truth version control pull/merge request flows triggers buildpacks build images autonomous rebuilds patch OS at scale gitops declarative desired state continuous operations tekton pipelined workflows
  6. 10 • CNCF Sandbox project initiated by Pivotal and Heroku

    in 2018 ◦ Contributors include Google, Microsoft, and others • Well-defined platform-to-buildpack contract to take source code and output OCI images • Focus on how to orchestrate builds, not how to run an app • Unified standard to enable ecosystem of platforms and modular buildpacks • Optimize for fast rebuilds, minimal data transfer, and efficient OS rebasing Cloud Native Buildpacks (CNB)
  7. pack kpack Spring Boot Tekton Tanzu Build Service az acr

    pack Google Cloud Run Button builder image buildpacks pla$orm run image * Platform Ecosystem Buildpacks Ecosystem Cloud Native Buildpacks (CNB)
  8. 12 kpack: Buildpacks as a Service • Automatically rebuild images

    based on changes in inputs • Efficiently rebase OS layers with a CVE hotfix (new run-image) • Consistent, repeatable, transparent and secure builds -- at scale Hosted, Centralized, Automated
  9. 13 Primarily a Package Manager (Charts), used for configuration customization

    as well Parameterized kubernetes resources + values.yml files for different environments Lots of {{-if / else}} switches The Kubernetes Package Manager Helm • Excellent for deploying OTS software • Unreadable GoLang templating (example) • Mandatory semver • Not great for CD • Non-declarative by default
  10. 14 Does one thing well Cleanly separates static resources (fixed),

    the resource changes (overlay), and the config that combines them (kustomize) Kubernetes native configuration management Kustomize • ConfigMap lifecycle mgmt • Part of kubectl • Simplicity • Transformers • Not every part of the resources are “selectable” for changes
  11. 15 #1. The entire system described declaratively. Kubernetes is just

    one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self-healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Four Principles of GitOps GitOps
  12. 16 INSTRUCTIONS 1. From city center, turn onto Route 4

    north. 2. At the traffic circle, take the first exit 3. At the first light, turn right. 4. Take the first available right turn. 5. At the traffic circle, start in the left lane and take the third exit. 6. The library is the first building on that road. How to get to the library? Imperative vs. Declarative FACTS • The library is located at 10 Community Way East Greenbush, NY 12061 United States uninvolved instructor, smart engine can recover from any starting point smart instructor, dumb engine must begin at starting point
  13. 17 IMPERATIVE • kubectl create … • kubectl run …

    • kubectl expose … • kubectl edit … • kubectl patch ... • kubectl label … • kubectl scale … • kubectl annotate … How to get to the desired state? Imperative vs. Declarative DECLARATIVE • kubectl apply … Controllers are the smart engines that converge running resources to desired state ✓ Great for creating reproducible deployments ✓ Great for creating yamls, troubleshooting, learning, experimentation
  14. 18 #1. The entire system described declaratively. Kubernetes is just

    one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self- healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Four Principles of GitOps GitOps
  15. 19 ➔ Code vs configuration ➔ Separation of access, cleaner

    audit logs ➔ Enables modular pipelines (“elbow joints”) ➔ Trigger action based on different life cycles ➔ Makes sense for coupled deployments comprised out of multiple repositories Separate repositories
  16. 20 #1. The entire system described declaratively. Kubernetes is just

    one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self-healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Principles of GitOps GitOps
  17. 21 #1. The entire system described declaratively. Kubernetes is just

    one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self- healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Principles of GitOps GitOps
  18. 22 #1. The entire system described declaratively. Kubernetes is just

    one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self-healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Principles of GitOps GitOps
  19. 23 Concept of Application (set of Kubernetes resources) Uses CRDs

    to store this information Focused on enterprise application deployments Provides UI with enterprise authentication capabilities ArgoCD
  20. 24 Doesn’t have the Application context, Flux is used primarily

    to bootstrap clusters where the application layer hasn’t necessarily been deployed yet Can scan image repositories for new versions and update the cluster automatically Intentionally doesn’t provide a UI Weave Flux
  21. 25 GitOps Promotions ✅ One repo for all pipelines ⛔

    Three separate pipelines in source repo ⛔ Possibly no notification when deploy doesn't complete
  22. 27 GitOps Promotions ✅ One repo for all pipelines ✅

    One pipeline to rule them all ✅ Guaranteed pipeline failure when deploy doesn't complete ⛔ The PRs keep the source pipeline execution waiting until merged
  23. 29 GitOps Promotions ⛔ Three separate repos for all pipelines

    ⛔ Three separate pipelines in three repos ✅ Guaranteed pipeline failure when deploy doesn't complete
  24. 31 GitOps Promotions ✅ Only one deployment repo to track

    ⛔ Three separate pipelines ✅ Guaranteed pipeline failure when deploy doesn't complete ⛔ No RBAC controls
  25. 33 Jenkins Pipeline - Stage - Step Concourse Resource -

    Job - Task Spinnaker Stage - Step - Task CircleCI Workflow - Job - Step Workflow Automation GitLab CI Stage - Job Travis Stage - Job ⬤ ⬤ ⬤
  26. 34 Jenkins Pipeline - Stage - Step Concourse Resource -

    Job - Task Spinnaker Stage - Step - Task CircleCI Workflow - Job - Step Workflow Automation GitLab CI Stage - Job Travis Stage - Job Tekton Pipeline - Task - Step ⬤ ⬤ ⬤
  27. 35 Generic workflow engine utilizing CRDs, happens to be really

    good at CD with Kubernetes Standardization of pipelines native to Kubernetes Provides other building blocks than pipelines as well Kubernetes-native Workflow Engine Tekton
  28. 36 Designed to run on and target Kubernetes Quickstarts &

    buildpacks Uses Prow (Lighthouse), Draft and Tekton internally Supports promotion flows with on-demand environments Hides Tekton’s complexity, but provides its flexibility Jenkins Kubernetes edition Jenkins X
  29. 39 Powerful turn-key multi-cloud deployment tool Advanced deployment strategies including

    automated canary analysis Keeps track of state of your deployments and uses it to drive pipelines Offers composite UI across clouds, clusters, regions and deployment platforms Complex to set up and a platform on its own to maintain Lacks declarative configuration of pipelines Deployment platform for fast and safe deployments Spinnaker
  30. 40 GitLab is a comprehensive SDLC platform, containing a Git

    server, an image registry, issue tracker, application monitoring, security scanning, … The differentiating feature is Auto DevOps It automatically sets up the complete pipeline and necessary integrations • detects the language of the code automatically builds • tests and measures code quality • scans for potential vulnerabilities, security flaws and licensing issues • packages • instruments (for monitoring in real-time) and deploys the application It provides an opinionated pipeline out-of-the-box Jenkins Kubernetes edition GitLab CI
  31. 41 Avoids boilerplate code which is the same for 80%

    of the projects by abstracting it. Examples of opinionated solutions: • Spring Boot provides an opinionated interpretation of the Spring framework abstracted in libraries • Cloud Native Buildpacks provide an opinionated way of building and containerizing your application • Spinnaker provides opinionated advanced deployment strategies • Jenkins X quickstart uses Draft packs to provide an opinionated way of bootstrapping your pipeline • GitLab’s Auto DevOps provides an opinionated full-featured end-to-end pipeline, including support for Cloud Native Buildpacks Characteristics Opinionated Pipelines
  32. package Path To Production change prod code test deploy test

    deploy promote rollback test strategic rollout reviews pull request feature branch declared state maintain test