EKS Cluster-based Canary Deploy implemented with Terraform Custom Provider

Transcript

1. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy Terraform Meetup Online 2021.02.10
2. 2 த઒ ஐӯ (Tomoaki Nakagawa ) Site Reliability Engineering Team @ free e Wor k ➔ EC2 ͔Β EKS ΁ҠߦɺࣗಈԽɺվળ Twitte r ➔ https://twitter.com/tmnkgwa4 GitHu b ➔ https://github.com/naka-gawa
3. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠓ೔࿩͢͜ͱ GSFFFͰ͸,VCFSOFUFT Ҏ߱&,4 ͸γϯάϧςφϯτઓུΛ औ͓ͬͯΓɺͦͷ݁Ռɺଟ਺ͷΫϥελΛ๊͑Δ͜ͱͱͳΓ· ͨ͠ɻͦͷӡ༻ͷதͰग़͖ͯͨ՝୊ײΛڞ༗͠ɺߴ͍Քಇ཰Λཁ ٻ͢ΔϓϩμΫτʹରͯ͠ɺͲͷ༷ʹ5FSSBGPSNΛ࢖͍ͭͭӡ༻Λ ࣗಈԽͨ͠ͷ͔Λ͝঺հ͠·͢ɻ
4. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy freeeʹ͓͚ΔΠϯϑϥCI/CDਤ ALB&TG EKS ./xxx-pj sg.tf alb.tf route53.tf ./share-pj iam.tf terraform-repo manifest-repo ./xxx-cluster cluster.yaml ./xxx-service helmfile.yaml ci ci ci
5. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ՝୊ -1- 
   ΫϥελͷόʔδϣϯΞοϓ
   
   
   
   ҆શʹ OP
   BMFSU ͰόʔδϣϯΛ্͛ऴ͑Δ·Ͱɺ13ඞཁʹͳ Δ
   
   
   
   
   
   SFQPTJUPSZ
   Λԟ෮ͨ͠Γɺ࡞ͬͨϦιʔεΛผ
   13
   Ͱ࡞ͬͨ Γ
   
   
   
   
   
   ΍Δ͜ͱ͸ఆܕ࡞ۀͳͷʹ΋͔͔ΘΒͣɺ߆ଋ͕࣌ؒ௕͘ͳΔ पลϦιʔεͷ؅ཧSFQPTJUPSZͱ&,4؅ཧSFQPTJUPSZ͕෼அ͞Ε͍ͯΔ
6. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ՝୊ -2- ߴ͍Քಇ཰Λཁٻ͞ΕΔΫϥελαΠζͷେ͖͍QSPEVDUʹର͢Δ੾Γ໭࣌ؒ͠ͷ௕ظԽ ALB&TG EK S 1.14->1.18 
   JOQMBDFͰWFSTJPO
   VQͯ͠ɺ໰୊͕͋ͬͨ ͔Β੾Γ໭͍ͨ͠৔߹औΓಘΔબ୒ࢶ͸৽ͨ ʹΫϥελ࡞੒͔͠ͳ͍ɻ
   
   
   
   Ϋϥελʹ"QQΛσϓϩΠ͢Δ·Ͱૣͯ͘ ෼ऑɺ4DBMFͷ໰୊΋ࢴ͢Δͱ࣌ؒ௒͑ Δ͜ͱ΋
   
   
   
   ࣌ظ࣍ୈͰ͸ݩʹ໭ͤͳ͍ͱ͍͏ࣄଶʹ ΋😇 EK S 1.14
7. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ՝୊ -3- େن໛ΫϥελʹͳΕ͹ͳΔ΄ͲɺXPSLFS
   OPEF
   ͷೖΕସ͑౳Ͱ΋Ұۤ࿑ 
   ࡉʑͱͨ͠ӡ༻ʹ͔͔Δ࣌ؒͷ௕ظԽ
   
   
   
   ۩ମྫΛ͋͛Δͱ
   
   
   
   8PSLFSʹ͚ͭΔ*".1PMJDZ΍4FDVSJUZ(SPVQ෇ସ͕൒೔࡞ۀ
   
   
   
   /PEF%SBJO͢Δʹ΋Ұ޻෉ඞཁ
8. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy Canary deploy per cluster ``্هͷΑ͏ͳมߋ͸௨ৗ
   JOQMBDF
   Ͱߦ͏ͱࢥ͍·͕͢ɺલऀʹ͍ͭͯ͸
   $POUSPM 1MBOF
   ͷ
   ,T
   ϚΠφʔόʔδϣϯΛҰ౓্͛ͯ͠·͏ͱԼ͛Δ͜ͱ͸Ͱ͖ͳ͍ɺޙऀʹ ͍ͭͯ͸໰୊ͷ͋ͬͨมߋΛ໭͚ͩ͢Ͱ΋਺ेඵఔ౓͕͔͔࣌ؒͬͯ͠·͏Մೳੑ͕͋ Γ·͢ɻ
   4-"తʹͦΕ͕ڐ͞Εͳ͍৔߹ʹɺԿ͔Ͱ͖Δ͜ͱ͸ͳ͍ͷͰ͠ΐ͏͔ʁ
   Ұͭͷํ๏͸ɺ"-#
   ΍
   /-#
   ͳͲͷޙΖʹෳ਺ͷ
   &,4
   ΫϥελΛ഑ஔ͠ɺҰํͷΫϥε λ΁ͷτϥϑΟοΫͷॏΈΛ
   
   ͳͲͷे෼ʹখ͍͞஋ʹͯ͠ɺͦͷॏΈͷখ͍͞Ϋϥ ελΛઌʹߋ৽͢Δ͜ͱͰ͢ɻͦͷޙɺಛʹΞϥʔτ౳্͕͕Βͳ͍ͷͰ͋Ε͹ɺͦͷม ߋ͸҆શͱΈͳͯ͠΋͏ҰํͷΫϥελʹશ͘ಉ͡มߋΛߦ͍·͢`` Ҿ༻ݩɿhttps://github.com/mumoshu/decouple-apps-and-eks-clusters-with-tf-and-gitops
9. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ ALB&TG EK S old ./xxx-pj sg.tf alb.tf route53.tf ./share-pj iam.tf terraform-repo manifest-repo ./xxx-cluster cluster.yaml ./xxx-service helmfile.yaml ci ci ci custo m provider EK S new weight rule
10. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ 5FSSBGPSN
    $VTUPN
    1SPWJEFS
    
    ఏڙ͞Ε͍ͯΔGSBNFXPSLΛར༻ͨ͠ಠࣗͷQSPWJEFS
    
    UFSSBGPSNQSPWJEFSFLTDUM
    
    UFSSBGPSNQSPWJEFSIFMNGJMF
    
    UFSSBGPSNQSPWJEFSLVCFDUM
11. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ UFSSBGPSNQSPWJEFSFLTDUM
    
    UFSSBGPSN͔ΒFLTDUMίϚϯ υΛ࣮ߦ͢Δத֩ͱͳΔ QSPWJEFS
    
    BMCSPVUFͷXFJHIUΛࣗಈ Ͱ੾Γସ͑ͯ͘ΕΔ΋ͷ provider "eksctl" { } resource "eksctl_cluster" "sample" { name = "sample " region = "ap-northeast-1 " spec = <<EO S nodeGroups : - name: ng 1 instanceType: m5.larg e desiredCapacity: 1 EO S }
12. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ UFSSBGPSNQSPWJEFSIFMNGJMF
    
    UFSSBGPSN͔ΒIFMNGJMFίϚ ϯυΛ࣮ߦ͢ΔQSPWJEFS
    
    ಉ࣮࣌ߦͯ͠΄͘͠ͳ͍ $SPO+PCͳͲ͸͜͜Ͱ੍ޚ͢ Δ provider "helmfile" { } resource "helmfile_release_set" "sample" { working_directory = "mamifest-repo " content = file("./ helmfile.yaml" ) binary = "helmfile " helm_binary = "helm " environment = "default " kubeconfig = eksctl_cluster.sample.kubeconfig_pat h }
13. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ UFSSBGPSNQSPWJEFSLVCFDUM
    
    σϑΥϧτϦιʔεΛ
    IFMN
    ؅ཧ͢ΔͨΊɺ BOOPUBUJPOMBCFM
    Λ෇༩͢Δ ͨΊͷ
    QSPWJEFS
    
    DPSFEOTͷύϥϝʔλΛҰ෦ ม͍͑ͨ৔߹ʹ࢖͍ͬͯΔ provider "kubectl" { } resource "kubectl_ensure" "sample" { kubeconfig = eksctl_cluster.sample.kubeconfig_pat h namespace = "kube-system " resource = "configmap " name = "coredns " labels = { "app.kubernetes.io/managed-by" = "Helm " } annotations = { "meta.helm.sh/release-name" = "coredns " "meta.helm.sh/release-namespace" = "kube- system " } } https://www.slideshare.net/ssuser4fddce/ekscoredns
14. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy Pros & Cons 1SPT
    
    ΫϥελόʔδϣϯΞοϓ࡞ۀʹ͓͍ͯΫϥελ࡞੒͔Β"QQ
    %FQMPZɺτϥϑΟοΫ੾Γସ͑·Ͱͷखॱ͕໿վળͨ͠
    
    ࠓ·Ͱݫີʹίʔυ؅ཧͰ͖͍ͯͳ͔ͬͨࡉ͔͍&,4Ϧιʔε͕ ίʔυ؅ཧͰ͖ΔΑ͏ʹͳͬͨ
    
    
    
    FH
    BXTBVUI
    
    *34"ͳͲͰ࢖͏*".SPMFͳͲͷϦιʔεͷࢀর͕Մೳʹ
    
    
    
    ࠓ·Ͱ͸సه͍ͯͨ͠
15. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy Pros & Cons $POT
    
    NBOJGFTU
    EFQMPZ
    DE
    ͸ͦͷ··։ൃऀ͕࢖͏ͨΊɺೋॏ؅ཧ ʹͳͬͯ͠·͏
    
    
    
    8PSLFS/PEF͚ͩมߋ͍ͨ͠ͷʹ"QQ%FQMPZ͕૸ͬͯ͠·͏
    
    $*ͷ౎߹্ɺ%FQMPZ IFMNGJMF
    TZOD ͕஗͍
    
    ΨνΨνʹݻΊΒΕͨ
    5FSSBGPSN
    NPEVMF
    ͕བྷΉͱͭΒ͍
    
    
    
    TUBHJOH
    
    QSPEVDUJPO
    ͱஈ֊తʹ্͍͕͛ͨ
    NPEVMF
    Λղ͘ ඞཁ͕͋Δ
16. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠓޙͷల๬ 
    ෼ք఺ͷௐ੔
    
    
    
    Ϋϥελ࡞੒͔Βɺ"QQ%FQMPZͰ͖Δঢ়ଶʹ͢Δ·ͰΛ 5FSSBGPSNͰߦ͍ɺ"QQ%FQMPZΛ"SHP$%ͳͲͷ%FQMPZʹಛԽ ͨ͠UPPMʹҠৡ
    
    ΑΓ1SPHSFTTJWF%FMJWFSZʹ͚ۙͮΔ
    
    
    
    "QQ%FQMPZޙͷΫϥελਖ਼ৗςετΛॆ࣮ͤ͞ɺࣗಈͰ੾Γ ໭ͬͨΓɺΫϥελ࡟আ·Ͱ΍ͬͨΓͰ͖ΔΑ͏ʹ͢Δ
    13
    Ͱ҆શ҆৺ʹΫϥελΛೖΕସ͑ΒΕΔੈք΁
17. εϞʔϧϏδωεΛɺ ੈքͷओ໾ʹɻ