Upgrade to Pro — share decks privately, control downloads, hide ads and more …

我が家のKubernetesクラスタ

Fumihiro Ito
December 14, 2022
180

 我が家のKubernetesクラスタ

Fumihiro Ito

December 14, 2022
Tweet

Transcript

  1. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster Who? • ҏ౻ ࢙ߒ • https://github.com/f110 •

    Z Lab Corporation • ϚωʔδυK8sαʔϏεͷ։ൃ • લ৬Ͱ͸εϚʔτϑΥϯ޲͚ήʔϜͷαʔόʔαΠυͷӡ༻ɾ։ൃ
  2. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster ͳͥࣗ୐ΫϥελΛߏஙͯ͠͠·ͬͨͷ͔ • K8sΤίγεςϜΛࢼͨ͠ΓϚχϑΣετΛॻ࣌͘ʹ࢖͑Δ؀ڥ͕ཉ͍͠ • minikube / kind

    ࣌୅ʢ2017ʙ2019೥ʣ • ࠷ॳ͸MacBook ProͰ • Ϧιʔε͕଍Γͳ͘ͳͬͨͷͰRyzen Threadripper 2950X΁ʢ16C32Tʣ • ׂͱແ஡Λͯ͠΋ಈ͘ɺҰ෦͚ͩʹো֐͕ൃੜ͢Δͱ͍͏͜ͱ͕ͳ͍ • ͦ͏ͩNUCͰΫϥελΛߏங͠Α͏
  3. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster ίϯηϓτ • ͦͦ͜͜K8sʹ৐ͤΔ • ͋Δఔ౓͸Ϋϥελ্Ͱಈ͔͍ͨ͠ • ͚ͲؤுΓ͗͢ͳ͍

    • ͿͬյΕ্౳ • յΕʢյ͠ʣͯम෮͢Δܦݧ͔ΒֶͿ΋ͷ͸ଟ͍ • ͪΐͬͱେࣄͳ΋ͷΛಈ͔͓ͯ͘͜͠ͱͰम෮ͷϞνϕʔγϣϯʹ͢Δ • ຊ౰ʹࠔΔ΋ͷ͸֎΁ग़͢
  4. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster $ kubectl get node whale1 -o jsonpath="{.metadata.creationTimestamp}"

    2019-10-21T02:59:56Z 2019೥ʹ2୆ߏ੒Ͱ࢝ΊͨΫϥελ ࠓͰ͸ίϯτϩʔϧϓϨʔϯx3 ϫʔΧʔx4ͷߏ੒ʹ $ kubectl get node NAME STATUS ROLES AGE VERSION rudder1 Ready control-plane 146d v1.24.3 rudder2 Ready control-plane 2y118d v1.24.3 rudder3 Ready control-plane 2y28d v1.24.3 whale1 Ready <none> 3y50d v1.24.3 whale2 Ready <none> 3y50d v1.24.3 whale3 Ready <none> 344d v1.24.3 whale4 Ready <none> 2y216d v1.24.3
  5. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster ϋʔυ΢ΣΞߏ੒ UniFi Dream Machine Pro UniFi Switch

    16 Control Plane Raspberry Pi 4 mem: 4GB disk: 64GB Raspberry Pi 4 mem: 4GB disk: 64GB Raspberry Pi 4 mem: 4GB disk: 64GB Worker Intel NUC Core i3-8109U Mem: 16GB Intel NUC Core i3-10110U Mem: 16GB Minisforum Ryzen 7 2700U Mem: 16GB Intel NUC Core i3-8109U Mem: 16GB
  6. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster ιϑτ΢ΣΞߏ੒ʢίϯτϩʔϧϓϨʔϯʣ rudder1 keepalived haproxy kube-apiserver etcd rudder2

    keepalived haproxy kube-apiserver etcd rudder3 keepalived haproxy kube-apiserver etcd kube-controller-manager kube-scheduler kube-controller-manager kube-scheduler kube-controller-manager kube-scheduler • ΦʔιυοΫεͳ haproxy / keepalived Λ࢖ͬͨ HA ߏ੒ • controller-manager ͱ scheduler ΋ಉډ • apiserver ΁͸ϩʔΧϧΞΫηε
  7. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster K8sج൫ ߏ੒πʔϧ kubeadm CNI calico ετϨʔδ Rook

    / Ceph ϩʔυόϥϯα MetalLB ίϯςφϨδετϦ Harbor γʔΫϨοτ؅ཧ Vault
  8. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster ϞχλϦϯά • Prometheus / Grafana / Grafana

    Loki ͷ Α͋͘Δߏ੒ • alertmanager ͸ Slack ʹ௨஌͢ΔͷΈ • Loki ʹϩάΛอଘ͍ͯ͠Δ͕΄ͱΜͲ ݟͯͳ͍ prometheus loki promtail node-exporter promtail alertmanager prometheus loki grafana grafana Slack kube-state-metrics
  9. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster GitOps • GitHub ʹϓϥΠϕʔτϦϙδτϦͱͯ͠ϚχϑΣετΛஔ͍͍ͯΔ • γʔΫϨοτ΋ϚχϑΣετͱͯ͠ೖΕ໊͍ͯͨ࢒ •

    git-crypt ͰϑΝΠϧࣗମ͸҉߸Խ͍͕ͯͨ͠ສ͕Ұͷ҉߸Խ๨ΕΛߟྀ͠ ͯϓϥΠϕʔτͰӡ༻ • ArgoCD ͰΫϥελʹద༻͢Δ
  10. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster ೔ʑͷӡ༻ • جຊతʹGitOps • γʔΫϨοτ͸Vaultʹอଘͯ͠argocd-vault-plugin • ࣗಈͰ൓ө͢Δ΋ͷͱࠩ෼Λ֬ೝͯ͠खಈͰ൓ө͢Δ΋ͷ͕͋Δ

    • PrometheusͰ؂ࢹͯ͠Ξϥʔτ͸Slack΁ • Ξϥʔτϧʔϧ͸গͳΊ • Ξϥʔτ͕ͳ͍ͱؾ͕෇͔ͳ͍Α͏ͳ΍ͭͷΈΛઃఆ͍ͯ͠Δ • ྫ͑͹ϊʔυͷCPU࢖༻཰ͳͲ͸؂ࢹ͠ͳ͍ • CPUϑΝϯͷԻ͕ฉ͑͜ΔͷͰҟৗ͸෼͔Δ
  11. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster ࣗ࡞Ingress controller • Zero Trust ProxyΛࣗ࡞͍ͯ͠ΔͷͰগ֦͠ுͯ͠Ingress controllerͱͯ͠΋ৼ

    Δ෣͏Α͏ʹͨ͠ • ৄ͘͠͸ https://bit.ly/f110-ingress-controller • IngressϦιʔε͸ೝূෆཁͰΞΫηεɺಠࣗϦιʔε͸ೝূͳͲ༷ʑͳઃఆ ͕Ͱ͖Δ • GitHubͷWebhookΛઃఆ͢ΔػೳͳͲ΋͋Δ
  12. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster όοΫΞοϓ • γϯϓϧʹetcdͷεφοϓγϣοτΛऔಘ͢ΔͷΈ • อଘઌ͸Google Cloud Storage

    • ετϨʔδͷσʔλ͸ఘΊΔ • όοΫΞοϓͷϓϩάϥϜ͸ࣗ࡞ • CronJobͰ3ִ࣌ؒؒͰ࣮ߦ • όοΫΞοϓϑΝΠϧ͸Ұिؒ෼ͷΈอ͍࣋ͯ͠Δ
  13. ςΩετ൛͸ 👉 https://bit.ly/f110-homecluster ΞϓϦέʔγϣϯ • ࣗ࡞Zero Trust Proxy • ࣗ࡞CI

    • Notion༻ࣗ࡞πʔϧ • ίʔυݕࡧπʔϧ • Թ౓ܭ༻ΤΫεϙʔλʔ