CloudNativeDaysTokyo 2019 slide

Transcript

1. KubernetesΛӡ༻ͨ͜͠ͱ ͰֶΜͩΞϯνύλʔϯ גࣜձࣾαΠόʔΤʔδΣϯτ ٕज़ຊ෦ Private Cloud Group
 ޙ౻ल৴ !1
2. Profile • ໊લ ޙ౻ ल৴ • ॴଐ ٕज़ຊ෦ PrivateCloudGroup ։ൃνʔϜ • ܦྺ 2014 / 04 ~ 2016/08 SIer 2016 / 09 ~ ݱࡏ ࡏ੶த • ۀ຿ Kubernetes, OpenShiftͷॾʑ !2
3. 1. ͜Ε·ͰͷPrivateCloudPlatform 2. Kubernetesͷߏ੒ 3. Ξϯνύλʔϯ8બ 4. ·ͱΊ !3
4. ͜Ε·Ͱͷ PrivateCloudPlatform !4
5. BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack Kilo
 Service VM BareMetal Kubernetes OpenStack Queens Service VM ͜Ε·ͰͷPrivateCloudPlatform 2013 - 2019 2015 - 2021 2019 - ݱࡏ΋Ұ෦ར༻த !5
6. BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack Kilo
 Service VM BareMetal Kubernetes OpenStack Queen Service VM 2013 - 2019 2015 - 2021 2019 - ݱࡏ΋Ұ෦ར༻த ͜Ε·ͰͷPrivateCloudPlatform !6
7. BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack Kilo
 Service VM BareMetal Kubernetes OpenStack Queen Service VM 2013 - 2019 2015 - 2021 2019 - ݱࡏ΋Ұ෦ར༻த ͜Ε·ͰͷPrivateCloudPlatform !7
8. ͳͥͦ͜ʹKubernetesʁ !8
9. BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack Kilo
 Service VM BareMetal Kubernetes OpenStack Queen Service VM 2013 - 2019 2015 - 2021 2019 - - 2013 ͜Ε·ͰͷPrivateCloudPlatform !9
10. • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ • όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ !10
11. ͜ͷΑ͏ͳ՝୊͕͋Γͭͭ
 ࣍ظPlatform͸Ͳ͏͠Α͏͔... !11
12. !12
13. ͋ͬͨ !13
14. !14
15. • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ • όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ !15
16. • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ • όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ DaemonSetͰղܾ !16
17. • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ • όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ DaemonSetͰղܾ ConfigMapͰղܾ !17
18. • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ • όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ DaemonSetͰղܾ ConfigMapͰղܾ ContainerͰղܾ !18
19. • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ • όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ DaemonSetͰղܾ ConfigMapͰղܾ ContainerͰղܾ Helm౳Ͱղܾ !19
20. Kubernetesͷߏ੒ !20
21. ৄࡉ৘ใ ,VCFSOFUFT  %PDLFS  6CVOUV  *OTUBMM ,VCFBEN $POUSPM
    1MBOF ୆ 8PSLFS
    /PEF ໿୆ !21
22. Control Plane Node - baremetal Worker Node (OpenStack: Compute Node) - baremetal Neutron-proxy Nova-compute Neutron- openvswitch VM Ceph Openstack master Kubernetes master Nginx-ingress etc... Prometheus Harbor fluentd / exporter VM VM Maxsacle GaleraCluster × 5 × 135 (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Control Plane Worker Node !22
23. 
 ՝୊Λղܾ !23
24. ͜͜·Ͱ͸ྑ͔ͬͨ
 (໿1೥൒લ) !24
25. ͠͹Βͯ͘͠ !25
26. !26
27. !27
28. ͱ͍͏͜ͱͰ !28
29. Ξϯνύλʔϯ8બ • ग़͘Θͨ͠ΞϯνύλʔϯΛ8ݸ΄Ͳ͓఻͑͠·͢ • όʔδϣϯʹΑͬͯ౰ͯ͸·Βͳ͍৔߹΋͋Γ·͢ ※ ղܾํ๏ͳͲ͸͋͘·ͰݸਓతͳݟղͰ͢ ※ ຊࢿྉʹܝࡌ͞Εͨ಺༰ʹΑͬͯੜͨ͡ଛ֐౳ͷҰ੾ͷ੹೚Λෛ͍͔Ͷ·͢ͷͰ
 ྃ͝ঝ͍ͩ͘͞. ※ ຊࢿྉʹܝࡌ͞Εͨ಺༰Λར༻͢Δ৔߹͸ࣗݾ੹೚Ͱ͓ئ͍͠·͢ !29
30. Ξϯνύλʔϯ8બ !30
31. Ξϯνύλʔϯ1
 Control PlaneʹͦͷଞPod΋૬৐Γ !31
32. • ͳͥͦ͏ͨ͠ʁ - ࠓճ͸OpenStackͷͨΊͷKubernetes
 ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ؅ཧͨ͘͠ͳ͔ͬͨ(ӡ༻తͳ໘Ͱ)
 (ϊʔυ਺ΛՄೳͳݶΓ཈͍͑ͨཉٻ͸Ͳ͜ʹͰ΋͋Δ͸ͣ) Kubernetes Control Plane Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Control Planeʹͦͷଞpod΋૬৐Γ !32
33. • Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes Control Plane Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !33
34. • Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes Control Plane Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Original App !34
35. • Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes Control Plane Nova-XXXX lube-apiserver Kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Original App !35
36. • Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes Control Plane Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !36
37. • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - Control PlaneʹͦͷଞͷPod͕৐ͤͳ͍Α͏ʹͨ͠ํ͕ྑ͍
 ؅ཧ༻ͷInfra Workerϊʔυ౳Λ༻ҙ͢Δ (σϑΥϧτͰ͸৐Βͳ͍) Kubernetes Control Plane Infra Worker Node Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !37
38. • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - Control PlaneʹͦͷଞͷPod͕৐ͤͳ͍Α͏ʹͨ͠ํ͕ྑ͍
 ؅ཧ༻ͷInfra Workerϊʔυ౳Λ༻ҙ͢Δ (σϑΥϧτͰ͸৐Βͳ͍) Kubernetes Control Plane Infra Worker Node Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !38
39. Ξϯνύλʔϯ2
 Ϋϥελ಺ͷϛχϚϜCeph !39
40. • ͳͥͦ͏ͨ͠ʁ - ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ؅ཧͨ͘͠ͳ͔ͬͨʢӡ༻తͳ໘Ͱʣ - osdͱ͔mon͕མͪͯ΋ࣗಈͰ෮چͯ͘͠ΕΔ - PVCͱ͔Ͱ࢖͍͍ͨͳ.....
 - CephΛߏங͢ΔVM͕͍ͳ͍
 - ΦϒδΣΫτετϨʔδͱͯ͠࢖͍͍ͨͳ.... Ϋϥελ಺ͷϛχϚϜCeph !40
41. • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr ࢒Γ
 ϝϯςத (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !41
42. • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack OpenStack ࢒Γ
 ϝϯςத !42
43. • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ
 Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack OpenStack ࢒Γ
 ϝϯςத !43
44. • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ
 - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ
 ϝϯςத !44
45. • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ
 - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ
 ϝϯςத !45
46. • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ
 - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ
 ϝϯςத !46
47. • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ
 - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ
 ϝϯςத !47
48. Infra Worker Node osd - A Ceph-mon Infra Worker Node Infra Worker Node osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr ɾɾN • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ϊʔυ਺͸ͨ͘͞Μ͋ͬͨํ͕ྑ͍ !48
49. Ξϯνύλʔϯ3 Ϋϥελ಺ͷPrivate Registry !49
50. • ͳͥͦ͏ͨ͠ʁ - Registryͷؔ࿈ίϯϙʔωϯτ͕མͪͯ΋࠶ىಈͯ͘͠ΕΔ͔Β͖ͬͱָ - CephͷPVC࢖͑͹؆୯Ͱ͸ʁʁ Kubernetes Control Plane Kubernetes 
 Control Plane XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX Kubernetes 
 Control Plane Ϋϥελ಺ͷPrivate Registry !50
51. Kubernetes Control Plane Kubernetes 
 Control Plane XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX Kubernetes 
 Control Plane • Կ͕ىͬͨ͜ - Registry͕ଘࡏ͢Δϊʔυ͕མͪͨ৔߹ɺRegistry͕෮چ͢Δ·ͰΠϝʔδΛϓϧͰ͖ͳ͍ ৔߹͕͋Γɺ௨ৗΑΓ෮چʹ͕͔͔࣌ؒΔ৔߹΋͋Δ - Ceph่͕յͨ͠λΠϛϯάͰ࢖͑ͳ͘ͳͬͨ !51
52. Kubernetes Control Plane Kubernetes 
 Control Plane XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX Kubernetes 
 Control Plane • Կ͕ىͬͨ͜ - Registry͕ଘࡏ͢Δϊʔυ͕མͪͨ৔߹ɺRegistry͕෮چ͢Δ·ͰΠϝʔδΛϓϧͰ͖ͳ͍ ৔߹͕͋Γɺ௨ৗΑΓ෮چʹ͕͔͔࣌ؒΔ৔߹΋͋Δ - Ceph่͕յͨ͠λΠϛϯάͰ࢖͑ͳ͘ͳͬͨ ͋Δϊʔυ͕μ΢ϯ
 (ϨδετϦͷؔ࿈ίϯϙʔωϯτ͕ଘࡏ) !52
53. • Կ͕ىͬͨ͜ - Registry͕ଘࡏ͢Δϊʔυ͕མͪͨ৔߹ɺRegistry͕෮چ͢Δ·ͰΠϝʔδΛϓϧͰ͖ͳ͍ ৔߹͕͋Γɺ௨ৗΑΓ෮چʹ͕͔͔࣌ؒΔ৔߹΋͋Δ - Ceph่͕յͨ͠λΠϛϯάͰ࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane Infra Worker Node kube-apiserver kube-controller kube-scheduler Registry Infra Worker Node XXXXXX XXXXXX ϨδετϦ͕෮چ͢Δ·Ͱ͸
 ϊʔυ্ʹPod͕࡞ΒΕͳ͍ XXXXXX XXXXXX Infra Worker Node XXXXXX XXXXXX Kubernetes 
 Control Plane Kubernetes 
 Control Plane Kubernetes 
 Control Plane !53
54. • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - k8sΫϥελͷ֎·ͨ͸ผk8sΫϥελʔͷ΋ͷΛར༻ͨ͠ํ͕ࣗ༝౓͸ߴ͍
 (Ҡઃ΋ߦ͍΍͍͢) - ৑௕͠΍͍͢RegistryΛ࢖͏ඞཁ͕͋Δ 
 (όοΫΤϯυʹDB͕͋Δ৔߹͸ͦΕΒ΋ؚΊͯ) Kubernetes Control Plane Infra Worker Node XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX ͋Δϊʔυ͕μ΢ϯ Kubernetes 
 Control Plane Kubernetes 
 Control Plane !54
55. • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - k8sΫϥελͷ֎·ͨ͸ผk8sΫϥελʔͷ΋ͷΛར༻ͨ͠ํ͕ࣗ༝౓͸ߴ͍
 (Ҡઃ΋ߦ͍΍͍͢) - ৑௕͠΍͍͢RegistryΛ࢖͏ඞཁ͕͋Δ 
 (όοΫΤϯυʹDB͕͋Δ৔߹͸ͦΕΒ΋ؚΊͯ) Kubernetes Control Plane Infra Worker Node XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX ଞͷϊʔυ্ʹpod͕࡞ΒΕΔ Kubernetes 
 Control Plane Kubernetes 
 Control Plane !55
56. Ξϯνύλʔϯ4
 Ϋϥελ಺ͷPrometheus !56
57. • ͳͥͦ͏ͨ͠ʁ - ؔ࿈ίϯϙʔωϯτ͕མͪͯ΋࠶ىಈͯ͘͠ΕΔ͔Β͖ͬͱָ - CephͷPVC࢖͑͹؆୯Ͱ͸ʁʁ Ϋϥελ಺ͷPrometheus !57
58. • Կ͕ىͬͨ͜ - Ceph → Prometheus ͜ͷॱ൪Ͱར༻ෆՄʹͳͬͨ - ౰࣌֎ܗ͔Βͷ؂ࢹ͕ͳ͍ͷͰɺ্هͷ໰୊͕ى͖ͯ΋੩͔ͩͬͨ • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ֎ܗ؂ࢹΛ͢ΔͳΓɺ֎͔ΒKubernetsͷAPIΛୟ͍ͯSDͰ؂ࢹ͢Δύλʔϯ͕ྑ͔ͬͨ ฐࣾͷத੢͕Prometheus Tokyo Meetup #2 ʹͯৄࡉ͸ൃදͯ͠·͢ (ࢿྉ) !58
59. Ξϯνύλʔϯ5
 େ͖ͳWorker NodeΛগͳ͍਺ͰͲʔΜ !59
60. • ͳͥͦ͏ͨ͠ʁ - ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ(ry • Կ͕ىͬͨ͜ - Ұ୆ʹ৐ΔPodͷྔ͕͔ͳΓଟ͘ͳΓɺҰ୆མͪͨͱ͖ͷΠϯύΫτ͕େ͖͘ͳͬͨ - ؾܰʹର৅ϊʔυͷϝϯςφϯεΛ͠ਏ͍ - Ұ୆མͪͨͱ͖ʹPodͷߦ͖ઌ͕ͳ͘ͳ͍ͬͯͨ • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ͋Δఔ౓খ͞ͳϊʔυΛͨ͘͞Μฒ΂ͨํ͕ྑ͍(೔ʑͷӡ༻ϝϦοτ͸͋Δ) େ͖ͳWorker NodeΛগͳ͍਺ͰͲʔΜ !60
61. Ξϯνύλʔϯ6
 ίϯςφͷϩάϩʔςΛ͍ͯ͠ͳ͍ !61
62. • ͳͥͦ͏ͨ͠ʁ - લఏͱͯ͠໰୊͕ى͖ͨͷ͸ComputeϊʔυͷϗετͰ
 VM༻ʹϦιʔεΛ֬อ͢Δ΂͘KubernetesͰ࢖͏Ϧιʔε͸ͳΔ΂͘࢖Θͳ͍Α͏ʹ͍ͯͨ͠
 ͕ϩάϩʔςͷ͜ͱΛ׬શʹࣦ೦͍ͯͨ͠ ίϯςφͷϩάϩʔςΛ͍ͯ͠ͳ͍ !62
63. • Կ͕ىͬͨ͜ - PodσϓϩΠ࣌ʹPodͷঢ়ଶ͕Pendingʹͳͬͨ (σϑΥϧτͷϩάϩʔςͩͱ଍Γͳ͍) • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - docker-logdriverͷઃఆΛద੾ʹ͢Δ΂͖(αΠζ΍ੈ୅؅ཧ) - ෆཁͳdebugϩάͳͲ͸ग़͞ͳ͍Α͏ʹ͢Δ !63
64. Ξϯνύλʔϯ7
 in_tailͱϩάϩʔςͷར༻ !64
65. • ͳͥͦ͏ͨ͠ʁ - ϩάͷංେʹΑΓPod͕εέδϡʔϦϯά͞Εͳ͍Α͏ʹͳͬͨͷͰ
 ϩάΛfluentdͰtailͭͭ͠΋1࣌ؒຖʹlogrotateΛಈ͔͍ͯͨ͠ in_tailͱϩάϩʔςͷར༻ !65
66. • Կ͕ىͬͨ͜ - 1࣌ؒͷස౓ͰlogrotateΛͯ͠ɺಉ࣌ʹfluentdͰ”*”Ͱin_tail͍ͯ͠Δͱ fluentd͕ߴස౓ͰLimitʹ౰ͨͬͯOOMͰkill͞ΕΔΑ͏ʹͳͬͨ ௚઀ؔ܎͋Δ͔͸ௐ΂͖Εͯͳ͍͕ɺ͜ͷΑ͏ͳissues΋͋Γ (https://github.com/fluent/fluentd-kubernetes-daemonset/issues/156) (https://github.com/fluent/fluentd/blob/master/lib/fluent/plugin/in_tail.rb#L255-L259) • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ϩάϩʔςΛ͢ΔλΠϛϯά͸௨ৗ௨Γʹ͠ɺੈ୅Λগͳ͘͢ΔΑ͏ʹ͢Δ - in_tailΛར༻͠ͳ͍ํ๏ͰؤுΔ !66
67. Ξϯνύλʔϯ8 ա౓ͳCronJobͷར༻ !67
68. • ͳͥͦ͏ͨ͠ʁ - όονॲཧΛຖ෼ߦͳ͍ͬͯΔ΋ͷ͕ෳ਺͍ͨ - ͦͷଞόοΫΞοϓͳͲ΋جຊతʹ͸CronJobͰճ͍ͯͨ͠ - ͜ΕΛControl PlaneͰճ͍ͯͨ͠ ա౓ͳCronJobͷར༻ !68
69. • Կ͕ىͬͨ͜ - Podͷ্͛Լ͕͛සൟʹߦΘΕΔ͜ͱͰɺϗετଆͷbuffer/cache͕ղ์͞Εͳ͍Ͱ free memory͕0%ʹͳΔ͜ͱͰϗετμ΢ϯ͕ൃੜͨ͠ ·ͨɺ/sys/fs/cgroup/memory഑Լ͕ҟৗʹͳ͍ͬͯͨ(Cannot Allocate Memory) - Control PlaneͰ͜ΕΒ͕ಈ͍͍ͯͨͨΊසൟʹControl Plane͕ෆ҆ఆʹͳͬͨ • Ͳ͏ͨ͠ํ͕ྑ͔͔ͬͨ - ৔౰ͨΓతͳରԠͱͳ͍ͬͯΔ͕ɺ ͦͷଞͷPodʹӨڹΛٴ΅͞ͳ͍Α͏ʹCronJobΛ࣮ߦ͢Δϊʔυ͸labelΛར༻ͯ͠ɺ CronJobՔಇઐ༻ͷWorker Nodeͱͯ͠ෳ਺୆Քಇ͍ͤͯ͞Δ !69
70. ະͩରԠதͷ෦෼΋͋Γ
 ӡ༻Ͱؤு͍ͬͯΔ ߲໨΋͋Γ·͢ !70
71. Ҏ্
 Ξϯνύλʔϯ8બͰͨ͠ !71
72. ·ͱΊ !72
73. ·ͱΊ • ౰࣌͸Kubernets͕͍͍ײ͡ʹ΍ͬͯ͘ΕΔͩΖ͏ͱґଘ͍͗ͯͨ͢͠ • Kubernetesͷࢥ૝͔Β͋·Γʹ΋ζϨͨ࢖͍ํΛ͢Δͱਏ͘ͳΔ • ӡ༻͕໘౗͔ͩΒͱ͍͏ࢹ఺͚ͩͩͱٯʹਏ͘ͳΔ • ͨͩɺKubernetesΛ̍೥΄Ͳӡ༻ͯ͠Έͯศརͳͱ͜Ζ΋
 ৭ʑͱײ͍ͯ͡Δ !73
74. ·ͱΊ • ౰࣌͸Kubernets͕͍͍ײ͡ʹ΍ͬͯ͘ΕΔͩΖ͏ͱґଘ͍͗ͯͨ͢͠ • Kubernetesͷࢥ૝͔Β͋·Γʹ΋ζϨͨ࢖͍ํΛ͢Δͱਏ͘ͳΔ • ӡ༻͕໘౗͔ͩΒͱ͍͏ࢹ఺͚ͩͩͱٯʹਏ͘ͳΔ • ͨͩɺkubernetesΛ̍೥΄Ͳӡ༻ͯ͠Έͯศརͳͱ͜Ζ΋
 ৭ʑͱײ͍ͯ͡Δ ༻๏ɾ༰ྔΛकͬͯద੾ʹ !74
75. ͋Γ͕ͱ͏͍͟͝·ͨ͠ !75