Share
Embed
コードを書くことに集中したい全てのアプリ開発者に贈るKubernetesの話 / Kubernetes Tips for all application engineers who want to focus on coding
コンテナネイティブな時代、アプリケーションエンジニアに求められる技術領域はさらに広がりを見せつつあります。インフラとアプリの境界を変えたコンテナ技術にどこまで踏み込めばいいのか迷っているアプリケーションエンジニアは多いと思います。このセッションでは、アプリケーションエンジニアがコーディングに集中するために知っておきたいコンテナ技術のおさらいから、Kubernetesとその周辺ツールの基本をデモも交えて説明します。(Japan Container Days v18.12)
Yoshiaki Mizukura
December 05, 2018
Transcript
54 slides
-
Japan Container Days v18.12 ίʔυΛॻ͘͜ͱʹूத͍ͨ͠શͯͷ ΞϓϦ։ൃऀʹଃΔKubernetesͷ Senior Architect Yoshiaki Mizukura
SIOS Technology, Inc. Dec 5, 2018 -
2 αΠΦεςΫϊϩδʔגࣜձࣾ ٕज़2෦ ΞϓϦέʔγϣϯίϯαϧςΟϯάάϧʔϓ γχΞΞʔΩςΫτ ਫ ྑ໌ About me •
Ϋϥυج൫த৺ͷΠϯϑϥߏங͔ΒΞϓϦ։ൃ • σʔλੳج൫ߏங • WebΞϓϦ։ൃʢΤϯλʔϓϥΠζܥʣ Powered by https://wordart.com/ -
3 • ΞϓϦ࣮ߦڥͷύοέʔδԽʹΑΔڥؒࠩҟͷۃখԽ • ίϯςφҎલɾɾɾ • ։ൃڥͰಈ͘ίʔυ͕ຊ൪ڥͰಈ͔ͳ͍ • ΞϓϦͱ࣮ߦڥͷߏཧͷ͢͠͞ •
Immutableͳ࣮ߦڥʹΑΔႈੑͷ୲อ ։ൃऀʹͱͬͯͷίϯςφϝϦοτ͓͞Β͍ ΞϓϦ֎෦ͷڥґଘʹ·͞ΕΔ͜ͱ͕ݮΓɺਝͳσϦόϦ͕ՄೳʹͳΔ -
5 ߏஙγεςϜ୯ҐͷมԽ γεςϜ͝ͱʹVMΛ࡞Δ γεςϜ͝ͱʹίϯςφΛ࡞Δ Ծج൫ H/W H/W H/W VM OS
M/W App Ծج൫ H/W H/W H/W M/W App M/W App M/W App M/W App ίϯςφج൫ VM OS VM OS VM OS VM OS M/W App VM OS M/W App VM OS M/W App VM OS ैདྷͷΠϯϑϥ͕ΞϓϦͱηοτʹͳͬͨʢΠϯϑϥͱΞϓϦͷڥքมԽʣ VMʢඇίϯςφʣ ίϯςφ -
6 ΠϯϑϥͱΞϓϦͷڥքมԽ Ծج൫ H/W H/W H/W M/W App M/W App
M/W App M/W App ίϯςφج൫ VM OS VM OS VM OS VM OS ίϯςφߏఆٛ ΞϓϦʹՃ͑ͯϛυϧΣΞɺωοτϫʔΫɺετϨʔδߏͷཧղ σʔλӬଓԽ ωοτϫʔΫ Docker Compose Kubernetes ႈͳ࣮ߦڥΛखܰʹߏஙͰ͖ΔΑ͏ʹͳͬͨ -
7 • ίϯςφͷϝϦοτཧղͰ͖Δͷͷɾɾɾ ΞϓϦ։ൃऀͷށ͍ ͦͦ։ൃͷྲྀΕͲ͏ͳΔͷʁ Logging CI/CD HTTPηογϣϯཧ σϓϩΠڥͷࠩҟ ࢹ
ϚΠΫϩαʔϏε͖ͱฉ͚͘Ͳɾɾɾ ϦτϥΠ ෛՙࢄ ࢄϩάͷτϨʔε ྲྀྔ੍ݶ αʔΩοτϒϨʔΧ োݕ αʔϏεؒͷґଘؔ ςετ -
10 • ίϯςφͷϝϦοτཧղͰ͖Δͷͷɾɾɾ ΞϓϦ։ൃऀͷށ͍ʢ࠶ܝʣ ͦͦ։ൃͷྲྀΕͲ͏ͳΔͷʁ Logging CI/CD HTTPηογϣϯཧ σϓϩΠڥͷࠩҟ ࢹ
ϚΠΫϩαʔϏε͖ͱฉ͚͘Ͳɾɾɾ ϦτϥΠ ෛՙࢄ ࢄϩάͷτϨʔε ྲྀྔ੍ݶ αʔΩοτϒϨʔΧ োݕ αʔϏεؒͷґଘؔ ςετ -
12 Delivery Pipeline base Image Image Coding Test Dockerfile /Manifest
Release Legacy Container-based खॱॻ Param Sheet build & deploy Code M/W M/W APP खॱॻ Param Sheet Code Manifest Dockerfile ڥߏங push ڥߏங ڥߏங ڥߏங ڥߏங build Image Registry Image Registry Code Repositry Code Repositry push pull push build push deploy deploy Manifest (for Test) Manifest (for Production) ڥߏங build & deploy ҟͳΔڥεϜʔζʹҠߦՄೳɻͲ͜ͰͪΌΜͱಈ͘ΞϓϦ Docker/Kubernetesͷ֓೦ɺؔ࿈CLIͷֶशίετ͔͔Δ (։ൃऀશһ͕શͯΛѲ͢Δඞཁͳ͘ϙΠϯτߜΕΔ) -
14 ઃఆϑΝΠϧ͕ڥมΛɾɾɾ • ಡΊΔ߹ • ManifestϑΝΠϧ͔ΒConfigMapΛࢀর͢Δ • ಡΊͳ͍߹ • ઃఆϑΝΠϧͷॻ͖͑
• ڥຖͷઃఆϑΝΠϧͰ্ॻ͖ • ςϯϓϨʔτΤϯδϯͰڥมΛຒΊࠐΉ σϓϩΠڥࠩҟͷΞϓϩʔν ίϯςφΠϝʔδʹڥґଘͷใΛอ࣋ͤͣɺڥมʹΓग़͢ -
15 • ઃఆใΛఆٛ͠ɺPodʹͤΔϦιʔε • SpringRailsɺઃఆϑΝΠϧͰڥมΛಡΈࠐΊΔͳ ΒConfigMap͕ద͍ͯ͠Δ • PodDeploymentଆͰڥมΛఆٛ͢Δ͜ͱՄೳɻ ϢʔεέʔεʹԠͯ͡બ ConfigMap
/ -2 : C/ " :1 C 2 2: 2 /1/ / :/ 2 C / 2 2 2 / C 2 / 2 2 : / :2 :/ 2 C / 2 / 2 C 2 2: :/ 2 C . / 2 : / 2 2 :/ 2 C / 2 2: : 2 C 2 :/ 2 C . / 2 : / 2 2 :/ 2 C / 2 2: : 2 C 2 / -2 : C :1 C : / 2 /1/ / :/ 2 C / 2 2: : 1/ / 2 C / 2 C / ConfigMap DeploymentϦιʔε͔Βͷࢀরํ๏ -
16 • envFromͰConfigMapΛ·ͱΊͯࢀর͢Δ͜ͱՄೳ ConfigMap Ωʔ໊ʹڥม໊Ͱఆٛ͢Δ / -2 : :1 :
/ 2 /1/ / :/ 2 / 2 2: : 1/ / . / . / ConfigMap / -2 : / " :1 2 2: 2 /1/ / :/ 2 / 2 2 2 / 2 / 2 2 : / :2 :/ 2 / 2 / 2 2 2: : / 2 :/ 2 / 2 2: : DeploymentϦιʔε͔Βͷࢀরํ๏ -
19 • ڥมΛຒΊࠐΊΔܰྔςϯϓϨʔτΤϯδϯ envsubst test.properties.tmpl >< = #> 1 .
> 1 Template Engine envsubst $ . test.properties • ڥมDeploymentConfigMapʹఆٛ • ίϯςφΠϝʔδଆͰenvsubstͯ͠ઃఆϑΝΠϧʹڥมΛຒΊࠐΉ $ $ . -
22 Logging App Logger rotate ϥΠϒϥϦϑϨʔϜϫʔΫػೳΛར༻ ैདྷʢඇίϯςφʣ ίϯςφ ίϯςφΤϯδϯͷlogging driverΛར༻
Docker logging driver App App stdout/err stdout/err somewhere (log store) ΞϓϦ͔Βඪ४/Τϥʔग़ྗϩάΛग़ྗ ϋϯυϦϯάlogging driverʹͤΔ -
23 දతͳυϥΠό ػೳ json-file JSONܗࣜͰग़ྗʢσϑΥϧτʣ ϩʔςʔγϣϯՄೳɻίϯςφআʹΑͬͯআ͞ΕΔɻ ϗετଆʹϑΝΠϧग़ྗ͞ΕΔɻ /var/lib/docker/containers/[container-id]/[container-id]-json.log syslog syslog
(※) ग़ྗ journald journald (※) ग़ྗ fluentd fluentd (※) ग़ྗ (forward input) awslog Amazon CloudWatch Logsग़ྗ gcplogs Google Cloud Platform Loggingग़ྗ Docker logging driver (※) ίϯςφ͔ΒଓՄೳͳσʔϞϯ͕ඞཁ -
24 • fluentd logging driverͰfluentdసૹ͠ɺElasticsearchू • kibanaͰՄࢹԽ ϩάूˍՄࢹԽͷྫ (Docker) fluentd
logging driver App App stdout/err stdout/err Docker store visualize Docker -
25 • ίϯςφ͚ͩͰͳ͘k8sίϯϙʔωϯτͷϩάؚΊͯfluentdͰू • DaemonSetͰfluentdΛશϊʔυʹඞͣ1Podͣͭஔ • ֤k8sϦιʔεͷlabelΛదʹઃఆ͓ͯ͘͜͠ͱͰɺElasticsearchͰͷݕࡧੑ্ • ϚΠΫϩαʔϏεͷ߹ɺαʔϏεϝογϡͷར༻ݕ౼ʢޙड़ʣ ϩάूˍՄࢹԽͷྫ
(k8s) store visualize logging driver App stdout/err logging driver App stdout/err (※) ؆ུԽͷͨΊIngress, Service, Podলུ k8s Node Docker Engine k8s Node Docker Engine k8s Node ৄࡉfluentdެࣜDaemonSetΛࢀর https://github.com/fluent/fluentd-kubernetes-daemonset -
26 • ΞϓϦ͔ΒͷϩάϑΝΠϧग़ྗͰͳ͘શͯඪ४ग़ྗΛ༻͢Δ • ϛυϧΣΞͷϩάඪ४ग़ྗʹͯ͠ίϯςφΠϝʔδԽ͢Δ • Dockerͷlogging driverJSONܗࣜͱ͢Δ͜ͱͰଐੑͰͷݕࡧ͕͠ ͘͢ͳΔ •
DeploymentͳͲk8sϦιʔεͷlabelΛదʹઃఆ͢Δ͜ͱͰݕࡧੑ Λ্Ͱ͖Δ Logging·ͱΊ -
Ͳ͜ʹηογϣϯσʔλΛอ࣋͢Δ͔ client client session client session session app session app
app app app app Stateful Stateless εέʔϧΞτ࣌ʹηογϣϯετΞͷՄ༻ੑΛߟྀ͢Δඞཁ͋Γ ཁSticky Session ʢηογϣϯಉظΛͯ͠Πϯελϯε͕૿͑ͨ߹ͳͲݶք͕͋Δʣ ηογϣϯແޮԽͷ੍ޚͳͲɺηΩϡϦςΟ໘ͷҙඞཁ Cookie Store (JWTηογϣϯσʔλอ࣋ʹ·͘͠ͳ͍) KVSͳͲ ՄೳͳݶΓΞϓϦ֎෦ʹอ࣋͠ɺΞϓϦΛεςʔτϨεʹ͓ͯ͘͠ͷ͕·͍͠ 28 -
29 • KubernetesͷSticky SessionػೳΛར༻ • Service (L4 Load Balancing) •
service.spec.sessionAffinityΛClientIPʹ͢Δ • Ingress (L7 Load Balancing) • NGINX Ingress Controllerͷ߹ • metadata.annotationsΛࢦఆ StatefulͳطଘΞϓϦΛվमͤͣʹks8্Ͱಈ͔͍ͨ͠ annotations: nginx.ingress.kubernetes.io/affinity: "cookie" nginx.ingress.kubernetes.io/session-cookie-name: "route" nginx.ingress.kubernetes.io/session-cookie-hash: "sha1" -
32 • ෳͷࢄαʔϏεؒͷ௨৴ • ϧʔςΟϯά • ϩʔυόϥϯγϯά • αʔϏεݸผͷϦϦʔε •
Blue/GreenσϓϩΠϝϯτ • ΧφϦΞϦϦʔε ߟ͑Δ͜ͱଟ͍ • োӨڹͷ࠷খԽ • αʔΩοτϒϨʔΧ • ϩάϝτϦΫεͷऩूɾࢹ • ࢄτϨʔγϯά -
33 • Netflix OSS • Eureka + RibbonɿαʔϏεσΟεΧόϦ • HystrixɿαʔΩοτϒϨʔΧ
• Spring CloudγϦʔζ ʑɾɾɾ ϥΠϒϥϦϑϨʔϜϫʔΫͰͷରԠॏ͍ ࣮ݴޠʹґଘ ղܾ͍ͨ͜͠ͱʹ߹ΘͤͯݸผʹϥΠϒϥϦબఆʢֶशίετͷ૿େʣ ΞϓϦέʔγϣϯͷຊ࣭Ͱͳ͍෦ʹ։ൃऀͷوॏͳϦιʔε͕औΒΕΔ -
34 • ॳɿࢄγεςϜʹ͓͚ΔαʔϏεؒ௨৴ • ͷ (mesh) ͷΑ͏ʹෳࡶʹབྷΈ߹͏௨৴ܦ࿏ • ۙɿαʔϏεؒ௨৴Λ੍ޚɾཧ͢ΔΈ •
τϥϑΟοΫཧ • ରোੑ • ςετ • ࢹɺʑ αʔϏεϝογϡ https://www.slideshare.net/BruceWong3/the-case-for-chaos -
35 • αʔϏεؒ௨৴Λ౷߹ཧ͢ΔιϦϡʔγϣϯ • Google/IBM/Lyftࣾ։ൃͷOSS • 2018/7 v1.0ϦϦʔε Istio ϚϧνϓϥοτϑΥʔϜରԠʢݱࡏKubernetesɺConsulʹରԠʣ
ΞϓϦέʔγϣϯίʔυͷΈࠐΈෆཁ ΞϓϦέʔγϣϯ࣮ʹґଘ͠ͳ͍ ࢹɺࢄτϨʔγϯάͳͲपลπʔϧͱͷੑ͕ߴ͍ -
36 • αʔϏεؒͷϓϩΩγͱͯ͠ಈ࡞ Istioಈ࡞Πϝʔδ Container App Service Container App Service
Kubernetes Container App Service Istio -
37 Istioػೳ • ϧʔςΟϯά • ϩʔυόϥϯγϯά • ϦτϥΠ/λΠϜΞτ/αʔΩοτϒϨʔΧ τϥϑΟοΫཧ •
ೝূɾೝՄ • αʔϏεؒ௨৴҉߸Խ ηΩϡϦςΟ • Logging • ࢄτϨʔγϯά • ֤छϝτϦΫεऩू ςϨϝτϦ -
38 IstioΞʔΩςΫνϟ • Proxyʹ֦ுͨ͠ Λ༻ • C++࣮ͷߴͳL4/7ϓϩΩγ • Graduated CNCF
Project • PodͷSidecarͱͯ͠ஔ͞ΕΔͨΊΞϓϦ ίʔυͷมߋෆཁ • PodϨϕϧͰDI͞ΕΔΠϝʔδ ProxyͷαʔϏε σΟεΧόϦ ςϨϝτϦ ೝূΩʔͷ ཧ ֤PodͷSidecar ͱͯ͠ஔ Data Plane Control Plane Pod Pod https://istio.io/docs/concepts/what-is-istio/#architecture -
39 Istio Demo client ࢹ Node Kubernetes Mixer ࢄτϨʔγϯά μογϡϘʔυ
Pod Pod Pod frontend postgresql rest-api Proxy Proxy Proxy ςϨϝτϦ Demoڥߏஙऀɿ αΠΦεςΫϊϩδʔ ख௩ -
40 • αʔϏεϝογϡʹΑͬͯࢄαʔϏεؒ௨৴Λ౷߹త ʹ੍ޚͰ͖ΔΑ͏ʹͳΔ • αʔΩοτϒϨʔΧͳͲΞϓϦଆͰࣗલ࣮͕ෆཁʹ • Istioػೳ͕๛ • ωοτϫʔΫपΓͳͲk8sͷػೳͱ߹ΘͤͯͲ͏ߏ͢Δ͔ͳͲ
શମΛ၆ᛌͨ͠ઃܭεΩϧ͕ٻΊΒΕΔ • ϚΠΫϩαʔϏεΛద༻͖͔͢Α͘ݕ౼͢Δ αʔϏεϝογϡ·ͱΊ -
42 kubectl logs • ίϯςφ୯Ґͷϩά֬ೝɻpod࠶࡞ͷʹίϚϯυଧͪ͢ඞཁ͕͋Δ stern • ϩάΛ֬ೝ͢ΔpodΛॊೈʹࢦఆՄೳ • ਖ਼نදݱlabelࢦఆɺnamespaceԣஅͰෳPodͷϩάΛtail
• pod࠶࡞Ͱid͕มΘͬͯਵͯ͘͠ΕΔ ޮతͳϩά֬ೝ $ $ $ - - - https://github.com/wercker/stern Deployment ReplicaSet Pod 1 Pod 2 Container A Container B Container A Container B kubectl logs stern -
43 • for Docker • Docker for Mac • https://docs.docker.com/docker-for-mac/#install-shell-completion
• Windows PowerShell༻͋Γʢະݕূʣ (※) bash on WindowsͰDockerΛ͏ख͋Γʢ͕ඞཁʣ • https://github.com/matt9ucci/DockerCompletion • for Kubernetes • kube-prompt • https://github.com/c-bata/kube-prompt ίϚϯυิ -
46 • freshpod • ϩʔΧϧΠϝʔδͷߋ৽Λݕͯ͠PodΛ࠶σϓϩΠͯ͘͠ΕΔ • freshpodࣗମpodͱͯ͠ಈ࡞ • MinikubeɺKubernetes on
Docker for Windows/MacͰར༻Մೳ k8s։ൃϫʔΫϩʔυͷޮԽʢΠϝʔδߋ৽ݕʣ freshpodʹΑΓࣗಈԽ͞ΕΔ෦ ߋ৽ ࣗಈݕ https://github.com/GoogleCloudPlatform/freshpod imagePullPolicyIfNotPresentͱ͢Δඞཁ͋Γ Coding Build Deploy Test -
freshpodʹΑΓࣗಈԽ͞ΕΔ෦ ߋ৽ ࣗಈݕ Coding Build Deploy Test 47 • entr
(Event Notify Test Runner) • ίʔυमਖ਼Λݕͯ͠ҙͷίϚϯυ࣮ߦ͕ՄೳͳCLI • freshpodͱΈ߹ΘͤΔ͜ͱͰίʔυमਖ਼ΛࣗಈతʹPodөՄೳ k8s։ൃϫʔΫϩʔυͷޮԽʢίʔυमਖ਼ݕʣ entrʹΑΓࣗಈԽ͞ΕΔ෦ http://eradman.com/entrproject/ *.' .$- . ' * ' - $ ίʔυอଘͷʹPod࠶ىಈ͕ൃੜ͢ΔͨΊ࡞ۀ༰ʹΑͬͯదٓ༻͢Δ ߋ৽ ࣗಈݕ -
Skaffold Pipeline 48 • Skaffold • σϓϩΠ·ͰͷύΠϓϥΠϯΛࣗಈԽ͢ΔGoogleCLIπʔϧ • ֤ϑΣʔζͷॲཧΛϓϥάΠϯతʹࠩସՄೳ •
ϩʔΧϧ։ൃதΠϝʔδpushෆཁͱͨ͠ΓɺσϓϩΠʹHelmΛར༻ͨ͠Γɺʑ • Πϝʔδͷλάࣗಈ༩ k8s։ൃϫʔΫϩʔυͷޮԽʢΑΓॊೈʹʣ Coding Build Deploy Test https://github.com/GoogleContainerTools/skaffold Push docker build docker push kubectl apply GA͍ͯ͠ͳ͍ͨΊɺޓੑͷͳ͍มߋʹҙ͕ඞཁ https://github.com/GoogleContainerTools/skaffold/blob/v0.19.0/deprecation-policy.md ߋ৽ ࣗಈݕ -
49 • ίϯςφΛ׆༻͢Δ͜ͱͰڥࠩҟʹۤ͠Ή͜ͱͳ͘ɺΞϓϦΛσϦόϦʔ Ͱ͖Δ • k8sͱΤίγεςϜ܈ߴػೳͰ͍͕͠ɺඞཁͳϙΠϯτΛߜΔ͜ͱͰֶ शίετԼ͛ΒΕΔ • k8sΤίγεςϜΛ׆༻͢Δ͜ͱͰ࣮ෛ୲Λ͑ͳ͕ΒෳࡶͰॊೈͳ ੍ޚΛ࣮ݱͰ͖Δ
• ϏδωεϩδοΫʹूதͰ͖Δ ʢϏδωεͷࠩผԽʹϦιʔεΛूதͰ͖Δʣ ·ͱΊ -
51 • Twelve Factor App by Heroku • https://12factor.net/ja/ •
Beyond the Twelve-Factor App by Pivotal • https://content.pivotal.io/blog/beyond-the-twelve-factor-app ʢࢀߟʣΞϓϦઃܭͷώϯτʹɾɾɾ -
52 • “ΠϯϑϥΤϯδχΞ͕k8sͰΞϓϦΛ࡞ͬͯݟ͑ͨࠓޙͷΠϯϑϥ”, SlideShare • https://www.slideshare.net/susumu0108/k8s-91891197 • “Istioͷશ”, Think IT.
• https://thinkit.co.jp/article/14640 • “IstioαʔϏεϝογϡೖ”, SlideShare • https://www.slideshare.net/yokawasa/istio-114360124 References