Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Goでkubernetes operatorを実装してアプリのプレビュー環境を作る/go-ku...

Goでkubernetes operatorを実装してアプリのプレビュー環境を作る/go-kubernetes-operator

fukuoka.go#17

Takuma Kume

June 10, 2021
Tweet

More Decks by Takuma Kume

Other Decks in Technology

Transcript

  1. zLVCFSOFUFTPQFSBUPSΛ࡞Δzͱ͸ apiserver kubectl apply Control plane Data plane controller manager

    ReplicaSet Pod Pod ࢦఆ͞ΕͨPodͷ਺Λ อͱ͏ͱ͢Δ ReplicaSetͷྫ replicas:2 PodΛ1ͭ࡟আͯ͠΋ ίϯτϩʔϥʔ͕ݕ஌ͯ͠ ࠶࡞੒͞ΕΔ
  2. zLVCFSOFUFTPQFSBUPSΛ࡞Δzͱ͸ apiserver kubectl apply Control plane Data plane controller manager

    ReplicaSet Pod Pod ࢦఆ͞ΕͨPodͷ਺Λ อͱ͏ͱ͢Δ ReplicaSetͷྫ replicas:2 PodΛ1ͭ࡟আͯ͠΋ ίϯτϩʔϥʔ͕ݕ஌ͯ͠ ࠶࡞੒͞ΕΔ kubernetes͸ ͜ͷಈ͖Λ֦ுͰ͖Δ kubernetesΛ֦ு͢Δख๏ͷͻͱͭʹOperator͕͋Δ
  3. zLVCFSOFUFTPQFSBUPSΛ࡞Δzͱ͸ apiserver kubectl apply Control plane Data plane Custom Controller

    Custom Resource Custom Resource Definition (CRD) + ಠࣗͷϦιʔεఆٛ CRDͷఆٛʹج͍ͮͨ Ϧιʔε CRΛίϯτϩʔϧͯ͠ ఆٛ͞Εͨঢ়ଶʹอͭ kubernetes operator ͷ࣮ମ
  4. apiVersion: service-expose.../v1alpha1 kind: ServiceExpose metadata: name: example namespace: ns1 spec:

    backend: service: name: example-svc port: number: 8080 domain: example.com path: / pathType: Prefix tlsEnable: true tlsSecretName: example-tls annotations: cert-manager.io/cluster-issuer: letsencrypt apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example namespace: ns1 annotations: cert-manager.io/cluster-issuer: letsencrypt spec: rules: - host: example-svc.ns1.example.com http: paths: - backend: service: name: example-svc port: number: 8080 path: / pathType: Prefix tls: - hosts: - example-svc.ns1.example.com secretName: example-tls backend: service: name: example-svc port: number: 8080 example-svc ns1 example.com example-svc ns1 example.com example-svc ns1 example.com ࢦఆͨ͠ServiceΛ IngressͰެ։͢Δ ެ։͢Δϗετ໊ͱͯ͠ αʔϏεσΟεΧόϦతʹ ࣗಈͰ෇༩͢Δ IUUQTHJUIVCDPNUBLVNBLVNFTFSWJDFFYQPTFPQFSBUPS αʔϏε໊ namespace υϝΠϯ Custom Resource
  5. kubernetes cluster app repo system manifests repo Pull Request Github

    Actions ArgoCD Config ArgoCD Github Actions ArgoCD Config Pod Ingress Namespace: app-pr-XXX 1.PRͷ࡞੒ 2.PRͷϒϥϯνΛ kubernetesΫϥελʹ σϓϩΠ͢ΔͨΊͷ ArgoCDͷઃఆΛੜ੒ commit Service 3.ArgoCDͷ ઃఆ௥ՃΛݕ஌ 4.ArgoCDͷઃఆΛ σϓϩΠ 5.PRͷϒϥϯνͷ σϓϩΠΛ։࢝ dispatch argocd-util ίϚϯυͰੜ੒ ʲ1VMM3FRVFTUຖͷϓϨϏϡʔ؀ڥʳ ArgoCD΍Github ActionsΛ׆༻ͯ͠Pull ReqτϦΨʔͰGitOpsͰϓϨϏϡʔ؀ڥΛੜ੒͍ͯ͠Δ 6.PRͷϒϥϯνͷ ϓϨϏϡʔ؀ڥ͕࡞ΒΕΔ 7. external-dnsͰAϨίʔυ, cert-managerͰTLSূ໌ॻΛ ࣗಈઃఆ
  6. !"" main.go #"" manifests !"" base $ !"" kustomization.yaml $

    !"" app.deployment.yaml $ #"" app.service.yaml #"" overlays !"" production $ !"" kustomization.yaml $ #"" app.ingress.yaml #"" staging !"" kustomization.yaml #"" app.ingress.yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: app-pr-XXX spec: destination: namespace: app-pr-XXX server: https://kubernetes.default.svc source: path: manifests/overlays/staging repoURL: https://github.com/takumakume/app targetRevision: future-branch syncPolicy: syncOptions: - CreateNamespace=true ֤؀ڥͷmanifestsΛkustomizeͰ؅ཧ͍ͯ͠Δ #"" staging !"" kustomization.yaml #"" app.ingress.yaml Pull RequestຖʹNamespaceΛ੾ͬͯ staging؀ڥͷෳ੡Λ࡞͍ͬͯΔ app-pr-XXX ʲ1VMM3FRVFTUຖͷϓϨϏϡʔ؀ڥʳ app repo
  7. ։ൃܦҢ  ٕज़త՝୊ apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: app-ingress

    namespace: app spec: rules: - host: staging-app.example.com http: paths: - backend: service: name: app-svc port: number: 8080 path: / pathType: Prefix tls: - hosts: - staging-app.example.com secretName: app-tls staging-app.example.com staging-app.example.com w ؀ڥͷෳ੡͸࡞Εͯ΋ɺ*OHSFTTͷϗετ ໊෦෼Λม͑Δ͜ͱ͕Ͱ͖ͳ͍ɻ w LVTUPNJ[Fͷ+40/1BUDIػೳ΍ɺZRίϚ ϯυͳͲͰஔ׵ͭͭ͠ద༻͢Δ͜ͱ͸Ͱ ͖Δ͕؅ཧ͕൥ࡶʹͳΔɻʢܦݧࡁʣ  w ద༻௚લͰNBOJGFTUTΛॻ͖׵࣮͑ͭͭ ߦ͢Δͱ(JU0QTʹΑΔԸܙ͕ബΕΔɻ ίί ίί
  8. ։ൃܦҢ  ࣮૷ํ਑ w (JU0QT͕Ͱ͖Δ͜ͱ w એݴతͰ͋Δ͜ͱ w ϓϨϏϡʔ؀ڥʹΞΫηε͢ΔͨΊͷϗετ໊ΛͲ͏͢Δ͔ʁ w

    LVCFSOFUFTͷ4FSWJDF%JTDPWFSZ w 4&37*$&@/".&/".&"1"$&TWDDMVTUFSMPDBM w *OHSFTTͰ࣮ݱͰ͖ΔͱศརͰ͸ͳ͍͔
  9. apiVersion: service-expose.../v1alpha1 kind: ServiceExpose metadata: name: example namespace: ns1 spec:

    backend: service: name: example-svc port: number: 8080 domain: example.com path: / pathType: Prefix tlsEnable: true tlsSecretName: example-tls annotations: cert-manager.io/cluster-issuer: letsencrypt apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example namespace: ns1 annotations: cert-manager.io/cluster-issuer: letsencrypt spec: rules: - host: example-svc.ns1.example.com http: paths: - backend: service: name: example-svc port: number: 8080 path: / pathType: Prefix tls: - hosts: - example-svc.ns1.example.com secretName: example-tls backend: service: name: example-svc port: number: 8080 example-svc ns1 example.com example-svc ns1 example.com example-svc ns1 example.com ެ։͢Δ Service αʔϏεσΟεΧόϦతͳ ϗετ໊ΛࣗಈͰ෇༩ αʔϏε໊ namespace υϝΠϯ ࠶ܝ
  10. kubernetes cluster app repo system manifests repo Pull Request Github

    Actions ArgoCD Config ArgoCD Github Actions ArgoCD Config Pod Namespace: app-pr-XXX 1.PRͷ࡞੒ 2.PRͷϒϥϯνΛ kubernetesΫϥελʹ σϓϩΠ͢ΔͨΊͷ ArgoCDͷઃఆΛੜ੒ commit Service 3.ArgoCDͷ ઃఆ௥ՃΛݕ஌ 4.ArgoCDͷઃఆΛ σϓϩΠ 5.PRͷϒϥϯνͷ σϓϩΠΛ։࢝ 6.PRͷϒϥϯνͷ ϓϨϏϡʔ؀ڥ͕࡞ΒΕΔ dispatch argocd-util ίϚϯυͰੜ੒ Service Expose ੜ੒ app.app-pr-XXX.example.com Ingress 7. external-dnsͰAϨίʔυ, cert-managerͰTLSূ໌ॻΛ ࣗಈઃఆ ServiceExposeͷΈσϓϩΠ͢Δ͜ͱͰIngress͸ࣗಈੜ੒͞ΕΔ
  11. 0QFSBUPS'SBNFXPSL  w ࣮૷ʹ͋ͨͬͯ΍ͬͨ͜ͱ w 0QFSBUPS'SBNFXPSLͷެࣜυΩϡϝϯτ͕ॆ࣮͍ͯ͠ΔͷͰɺج ຊతʹ͸ͦ͜Λࢀর͢Δ IUUQTTELPQFSBUPSGSBNFXPSLJPEPDTCVJMEJOHPQFSBUPSTHPMBOH  w

    ͨ·ʹࡌ͍ͬͯͳ͍৔߹͕͋ΔͷͰɺLVCFCVJMEFSͷެࣜυΩϡϝ ϯτͰิ׬͢Δ IUUQTCPPLLVCFCVJMEFSJP  w ϕετϓϥΫςΟεͷ࣮ફ IUUQTTELPQFSBUPSGSBNFXPSLJPEPDTCFTUQSBDUJDFTCFTUQSBDUJDFT 
  12.  w TFSWJDFFYQPTFPQFSBUPSͷ3FDPODJMFS-PPQ (PʹΑΔ0QFSBUPS࣮૷ Observe Diff Action - ੜ੒͢΂͖IngressͷSpec͸ʁ -

    ੜ੒͢΂͖Ingressͱݱࡏͷ Ingressͷࠩ෼͸ͳʹ͔ʁ - ࠩ෼ΛຒΊΔͨΊʹIngressͷ Create/Update/DeleteΛ࣮ߦ
  13.  w ςετ (PʹΑΔ0QFSBUPS࣮૷ ServiceExpose ঢ়ଶऔಘ Ingress͸ ଘࡏ͢Δ͔ʁ Ingressͷ Ξοϓσʔτ͸

    ඞཁ͔ʁ Ingress ੜ੒ Ingress Ξοϓσʔτ Y Y N N Reconciler Loop w ্هͷΑ͏ʹ0QFSBUPS͸ঢ়ଶભҠ͕ൃੜ͢Δɻ w ͋ΒΏΔύλʔϯͰ3FDPODJMFS-PPQ͕ႈ౳ʹͳΔΑ͏ʹςετ͠ ͍ͨɻ