Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introdution_to_PipeCD.pdf

nghialv
March 25, 2021

 Introdution_to_PipeCD.pdf

nghialv

March 25, 2021
Tweet

More Decks by nghialv

Other Decks in Technology

Transcript

  1. ࣗݾ঺հ - ৬ྺ @CyberAgent • PipeCDΛ։ൃɾӡ༻ - DPࣨ • Work

    fl ow Automation SystemΛ։ൃɾӡ༻ - OSSS • Feature Flags/Experimentation SystemΛ։ൃɾӡ༻ - AbemaTV • PrometheusͰMonitoring SystemΛߏஙɾӡ༻ - AbemaTV • Deployment ToolΛ։ൃɾӡ༻ - AbemaTV • Microservicesɾج൫पΓͷ࢓ࣄ - AbemaTV ΄ͱΜͲ͸ج൫΍ϓϥοτϑΥʔϜͷ͜ͱ 3
  2. CI != CD When people say “CI/CD,” they are only

    talking about Continuous Integration. 
 Nobody is talking about (or practicing) Continuous Deployment. AT ALL. 
 It’s like we have all forgotten it exists. It's time to change that. Charity Majors 8
  3. CI != CD Artifact Storage Verifying and Analysing the Impact

    Application Code 
 (.go, .java, .js...) Infrastructure Code 
 (.tf ...) Con fi guration Code 
 (.yaml ...) DockerHub, GCR, ECR... GCS, S3... Git Repository Code Storage Actions Continuous Integration Test Code Git Repository Host Environment Artifact Continuous Delivery Artifact Build and Save Artfacts Cloud User Low-risk actions including release strategy, rollback Deployment Dependency Management Provisioning, Installing Artifact 9 Artifact = Docker Image, Helm Chart, Kustomization Module, Terraform Module, ...
  4. Continous Delivery != Continuous Deployment Continuous Deployment means that every

    change goes through the pipeline and automatically gets put into production, resulting in many production deployments every day. In order to do Continuous Deployment we must be doing Continuous Delivery. Continuous Delivery just means that you are able to do frequent deployments but may choose not to do it, usually due to businesses preferring a slower rate of deployment. Artifact Continuous Deployment Dev, Test Env Artifact Continuous Delivery Prod Env An example 10 https://martinfowler.com/bliki/ContinuousDelivery.html
  5. Deploy != Release Deployment is the process for installing the

    new version of artifact on prod environment. 
 When we say a new version of software is deployed, we mean it is running somewhere in the production environment. Releasing is the process of moving production tra ff i c to the new version. When we say a version of a software is released, we mean that it is responsible for serving production tra ff i c. Deployment need not expose customers to a new version of your service. Given this definition, deployment can be an almost zero-risk activity. Turbine Labs 11 https://blog.turbinelabs.io/deploy-not-equal-release-part-one-4724bc1e726b
  6. PipeCDͱ͸ A uni fi ed continous delivery solution for multiple

    application kinds on multi-cloud A gitops tool that enables doing deployment operations by pull request on Git An open source project 12
  7. PipeCDͱ͸ 13 - A uni fi ed continous delivery solution

    for multiple application kinds on multi-cloud - A gitops tool that enables doing deployment operations by pull request on Git - An open source project
  8. PipeCD ❤ OSS 14 Thanks to the contributors of PipeCD!

    https://pipecd.dev/ https://github.com/pipe-cd/pipe https://pipecd.dev/docs/ - 2020/10݄ʹOSSͱͯ͠ϦϦʔε͠·ͨ͠ - 4ਓ͕ϑϧλΠϜͰPipeCD΁ίϛοτ͍ͯ͠Δ - 22 contributors͔Β1200 PRʹୡ੒͠·ͨ͠
  9. ͳͥPipeCDΛ࡞੒ͨ͠ͷ͔ Need of a uni fi ed delivery system Easy

    to operate multi-tenancy for multiple projects Easy to manage a large number of applications with a good DX Existing solutions do not fi t our requirements 15
  10. ౷ҰͳσϦόϦγεςϜ͕ඞཁ Project 1 CircleCI 16 Consistency Flexibility • ౷ҰͳγεςϜʹͳΔͱPlatform Team͕😊😊ɺDevelopers͕😊😥

    • ౷ҰͳγεςϜͰ͕͢ɺDevelopersͷFlexibilityͷอূ͕ඞཁ • ༷ʑͳΞϓϦέʔγϣϯछྨͷαϙʔτ͕ඞཁ • Kubernetes, Terraform, CloudRun, Lambda, ECS • GCP, AWS, Azure, Private Cloud • ࣗ෼Ͱ࣮૷͢ΔϩδοΫͰ΋ಈ͚Δ • ͲͷϓϩδΣΫτɾνʔϜͰ΋ϫʔΫ͢Δ͜ͱ͕ඞཁ • Ͳͷن໛Ͱ΋ϫʔΫ (3ਓνʔϜ͔Β100ਓνʔϜ·Ͱ) • νʔϜؒʹҠಈ࣌ͷΦϯϘʔσΟϯάίετ͕ແ͠ Project 2 Manually Project 4 FluxCD Project 25 Terraform Cloud + AWS Code Deploy + ArgoCD Project 3 Spinnaker ... Have to fi nd a good balance લͷঢ়ଶ
  11. Multi-Tenancyͷӡ༻͠΍͍͢΋ͷ͕ඞཁ 17 • ωοτϫʔΫͷ੍ݶνʔϜ΋αϙʔτඞཁ • Private cloudͳͲɺ֎͔Βͷ௨৴੍͕ݶ • SecretσʔλΛνʔϜͷΫϥελͷ֎ʹஔ͔ͳ͍ •

    RBACɾACLͷίϯτϩʔϧ͠΍͍͢ • Platform TeamͱDevelopersͷ໾ׂͱ͸͖ͬΓ෼ׂ • Platform Team͸γεςϜӡ༻ɾϓϥΫςΟεΛීٴ • Developers͸ར༻ɾϑΟʔυόοΫ 25 projectsҎ্
  12. طଘͷιϦϡʔγϣϯ͕ຬͨ͞ͳ͍ 18 ӡ༻ͷେม͞ ֶशίετ GitOpsͰ͸ͳ͍ʢඞਢͰ͸ͳ͍͕😊ʣ Visibilityͷ໰୊ (UIͳ͠ͳͲʣ Kubernetes ApplicationͷΈ
 Multi-Tenancyӡ༻Ͱ଍Γͳ͍

    ඪ४ͳDeploymentͷ୅ΘΓʹɺRollout CRDʹมߋඞཁ Kubernetes ApplicationͷΈ
 Multi-Tenancyӡ༻Ͱ଍Γͳ͍ Developer͕୭Ͱ΋ࣗ෼ͷαʔϏεΛߴ଎ɾ҆શɾ ҆৺ͰσϓϩΠͰ͖Δ (σϓϩΠதʹkubectlΛશ͘࢖Θͳ͍͍ͯ͘😊ʣ
  13. PipeCDͰͰ͖Δ͜ͱ Quick Sync and Progressive Sync Automated Rollback Automated Deployment

    Analysis Con fi guration Drift Detection
 Secret Management
 Event Watcher Noti fi cation 19
  14. Quick Sync vs Progressive Sync 20 Sync GitOpsͷҙਤ Quick Sync͸Clusterͷঢ়ଶΛGitͷঢ়ଶΛಉظ͢ΔͨΊʹɺ͙͢Gitͷঢ়ଶ΁ભҠ

    Progressive Sync͸Clusterͷঢ়ଶΛGitͷঢ়ଶΛಉظ͢Δ్தʹɺઓུ (canary, bluegreen, analysis...)ʹΑΓ
 ͍͔ͭ͘ͷதؒঢ়ଶʹܦ༝͢Δɻ͔͠͠ɺ࠷ޙతʹ͸Gitͷঢ়ଶʹભҠ Git Cluster Sync Sync
  15. Quick Sync vs Progressive Sync 21 Sync Quick Sync͸Clusterͷঢ়ଶΛGitͷঢ়ଶΛಉظ͢ΔͨΊʹɺ͙͢Gitͷঢ়ଶ΁ભҠ Progressive

    Sync͸Clusterͷঢ়ଶΛGitͷঢ়ଶΛಉظ͢Δ్தʹɺઓུ (canary, bluegreen, analysis...)ʹΑΓ
 ͍͔ͭ͘ͷதؒঢ়ଶʹܦ༝͢Δɻ͔͠͠ɺ࠷ޙʹ΋Gitͷঢ়ଶʹભҠ Git Cluster Sync Sync GitʹApplication directoryʹ.pipe.yamlͰArtifactͷύεɾσϓϩΠख๏ͳͲΛఆٛͰ͖Δ GitOpsͷҙਤ
  16. Configuration Drift Detection 26 https://pipecd.dev/docs/user-guide/con fi guration-drift-detection/ • ࣮ࡍͷঢ়ଶ͕ظ଴ͷঢ়ଶͱဃ཭ •

    Ϣʔβʔ͕௚઀ௐ੔ • ଞͷαʔϏε͕௚઀ௐ੔ • ࣗಈతʹCon fi guration DriftΛݕ஌ • WebUIͰࠩ෼Λදࣔ • ௨஌ͰΞϥʔτͷઃఆ͕Մೳ • ݱࡏ͸Con fi guration Drift͕ൃੜ͢Δͱɺ উखʹApply͠ͳ͍
  17. Secret Management 27 https://pipecd.dev/docs/user-guide/sealed-secrets/ • GitOps͸શͯͷ΋ͷΛGitʹอଘ • SecretΛ҆શʹอଘํ๏͕ඞཁ • PipeCD͸built-in

    secret؅ཧํ๏Λ࣋ͭ • Piped agent͕ར༻͢Δલʹ෮ݩΛߦ͏ 1 2 PipeCD webͰSecretͷ҉߸ԽΛߦ͏ ҉߸Խ͞ΕͨσʔλΛGitʹஔ͘ https://blog.stormcat.io/post/pipecd-sealed-secret/
  18. Event Watcher 28 FluxCDͷImage Updateػೳͷઆ໌ https://toolkit. fl uxcd.io/guides/image-update/ Container Registry

    Git Repository ArgoCD 
 FluxCD Watches images Makes commit to update image tags • GitOps͸શͯͷoperation͕Git PRΛ௨ͯ͠΍Δݪଇ • ৽͍͠container image͕Ͱ͖ͨΒɺࣗಈతʹGitΛߋ৽
  19. • GitOps͸શͯͷoperation͕Git PRΛ௨ͯ͠΍Δݪଇ • ৽͍͠container image͕Ͱ͖ͨΒɺࣗಈతʹGitΛߋ৽ • ͜ͷΞϓϩʔνͷ໰୊఺ • CI͔ΒCD΁౉͢Artifact͸Container

    Image͚ͩͰͳͳ͘ • Helm Chart • Kustomization Module • Terraform Module • Etc • աڈͷImage਺͕ଟ͍৔߹ʹRegistryͷWatchͷύϑΥʔϚϯε Event Watcher 29 Container Registry Git Repository ArgoCD 
 FluxCD Watches images Makes commit to update image tags
  20. Event Watcher 30 https://pipecd.dev/docs/user-guide/event-watcher/ pipectl event register \ --name=helloworld-image-update \

    --data=gcr.io/pipecd/helloworld:v0.2.0 apiVersion: pipecd.dev/v1beta1 kind: EventWatcher spec: events: - name: helloworld-image-update replacements: - file: helloworld/deployment.yaml yamlField: $.spec.template.spec.containers[0].image spec: containers: - name: helloworld - image: gcr.io/pipecd/helloworld:v0.1.0 + image: gcr.io/pipecd/helloworld:v0.2.0 • PipeCDͰ͸ Image Watcher ΑΓ Event WatcherػೳΛఏڙ • pipectlͰeventΛૹΔ͜ͱͰɺeventʹΑΓGitΛࣗಈతʹम ਖ਼ͯ͘͠ΕΔઃఆ͕Մೳ GitͷதʹeventʹΑΓमਖ਼ͷఆٛ Piped agent͕GitΛमਖ਼ͯ͘͠ΕΔ CIͰ೚ҙͷ࣌ؒͰeventΛൃੜ
  21. Notification 31 https://pipecd.dev/docs/operator-manual/piped/con fi guring-noti fi cations/ • ௨஌ઌͷઃఆ͕Մೳ •

    Slack • Webhook • ௨஌Πϕϯτͷઃఆ͕Մೳ • Deploymentͷ࣮ߦঢ়ଶ • Con fi guration drift͕ൃੜ • Application Healthͷঢ়ଶ • Pipedͷঢ়ଶ • etc
  22. ୯ҰͳΠϯλϑΣʔεɾ୯Ұͳϓϩηε 32 શͯͷػೳ͕Kubernetes, Terraform, Lambda, CloudRun, ECSͰ࢖͑Δ GCP, AWS, AzureͳͲcloud

    providerΛαϙʔτ Prometheus, Datadog, CloudWatch, Stackdriver LoggingͳͲͷσʔλͰ෼ੳΛߦ͑Δ
  23. νʔϜͱγεςϜͷߏ੒ 34 • Platform Team • GCP্Ͱશࣾ༻Control-PlaneΛӡ༻ • GCPͷFirestore &

    GCSͷϚωδʔυαʔϏεΛར༻ • StatelessͷServer & Cache͸K8sͷ্ʹಈ͘ • ֤ProjectͷSREs • Single binaryͷPiped agentΛΠϯετʔϧ • K8s cluster or Fargate or VMͷதʹಈ͘ • ֤ProjectͷDevelopers • WebͰ࢖͏ • GitͰPRΛૹͬͯɺσϓϩΠΛߦ͏
  24. ಋೖαʔϏε਺͕૿Ճத 0 100 200 300 400 2020/10 2020/11 2020/12 2021/01

    2021/02 2021/03 332 Applications/Services ʢ࢒ΓͷϓϩδΣΫτ΋Ҡಈதʣ 35
  25. ಋೖͰΑ͔ͬͨ͜ͱ 36 • Platform Team 😊 • શͯͷνʔϜͷσϓϩΠϝϯτΛ౷ҰͰ؅ཧ • ϓϥΫςΟεΛ࠾༻ɾීٴ͠΍͍͢

    • ӡ༻ָ͕ • શࣾͷ֤νʔϜ͔ΒϑΟʔυόοΫΛ΋Β͑Δ • Developers 😊 • kubectlͳͲ͕ෆཁͰɺߴ଎ɾ҆શɾ҆৺ͰσϓϩΠ • શͯͷσϓϩΠϝϯτ͕୯ҰͳΠϯλϑΣʔεɾ୯Ұͳϓϩηε • ৽نͷϓϩδΣΫτɾαʔϏεͷಋೖ͕͸΍͍ • ΦϯϘʔσΟϯάίετ͕௿͍ Consistency Flexibility Good Balance
  26. ࠓޙͷϩʔυϚοϓ • VisibilityΛ޲্ • Insights: Lead Time, Deployment Frequency, MTTR,

    Change Failure RateͳͲΛՄࢹԽ • Applicationͷঢ়ଶΛϦΞϧλΠϜతʹՄࢹԽ • Stage LogΛΑΓΘ͔Γ΍͘͢ɺ໰୊Λ͙͢ݟ͑ΔΑ͏ʹ • Multi-Provider, Multi-Tenancy • ECSͷαϙʔτ • ACLͰਂ͍Ϩϕϧͷݖݶ؅ཧ • Automated Deployment Analysis: CloudWatch, Stackdriver Logging... • AWS App Mesh, SMI • Secret Management • Sealed secretҎ֎ʹKMS, Vault΋αϙʔτ • ࣗ༝౓Λ্͛ΔͨΊʹɺϢʔβʔͷ࣮૷ͷpluginΛ࣮ߦͰ͖ΔΑ͏ʹ 38