ニコニコ生放送におけるWebフロントエンドBFFサーバーのKubernetes移行事例の紹介

2022/06/13に「ニコニコ生放送 WebフロントエンドのKubernetes移行ハンドブック 2022」を公開しました。本発表ではハンドブックの内容とともに、WebフロントエンドにおけるKubernetesへの移行と運用がどのようなものか紹介します。

Playground

Transcript

1. χίχίੜ์ૹʹ͓͚Δ WebϑϩϯτΤϯυBFFαʔόʔͷ KubernetesҠߦࣄྫͷ঺հ Kubernetes Meetup Tokyo #52 2022/08/17 Wed 19:00 ʙ Online
2. Name • Himeno Kosei Recent Career • 2022/07 - גࣜձࣾϋΠϠʔϧʔ • 2018/06 - 2022/06 גࣜձࣾυϫϯΰ Focus 1. Web Frontend 2. Go / Kubernetes / LB / Proxy GitHub • https://github.com/Himenon ࣗݾ঺հ
3. ໨࣍ 1. Introductionʢʙ 3minʣ 2. ҠߦϋϯυϒοΫͷ঺հʢʙ 3min ʣ 3. എܠઆ໌ͱ՝୊ʢʙ3minʣ 4. TypeScriptͰManifestΛॻ͘ʢʙ3minʣ 5. ArgoCD + SlackBotʢʙ3minʣ 6. ݕূ࡞ۀʢʙ 5minʣ 7. ΫϥελʔͷҠߦʢʙ 2minʣ 8. ࣭ٙʢʙ remain timeʣ a. Handbookதͷ಺༰Ͱ΋Մ
4. ಡऀ૚ ࣋ͪؼΓͰ͖Δͱࢥ͏΋ͷ KubernetesΛීஈ৮͍ͬͯΔਓ • ManifestΛTypeScriptͰӡ༻͢Δํ๏͕͋ΔΒ͍͠ ◦ ීஈ׳Ε͍ͯΔݴޠͰॻ͘͜ͱ΋Ͱ͖Δ • ൚༻ݴޠͰManifestΛੜ੒͢ΔͱYAMLΛॻ͘ͱຊ࣭తͳ࡞ۀʹूதͰ͖Δ TypeScriptΛීஈར༻͍ͯ͠Δਓ • KubernetesͷManifest͸ܕ҆શʹॻ͘͜ͱ΋Ͱ͖Δ • NodejsपΓͷΤίγεςϜʹ৐ΕΔ ͍Ζ͍Ζৄ͍͠ํʑ • ൚༻ݴޠΛ࢖ͬͨݕূαΠΫϧͷߴ଎Խͷํ๏ ຊൃදͰ࣋ͪؼΕΔ΋ͷ
5. 2021೥6݄͔Β2022೥3݄ͷ9ϲ݄ؒͰχίχίੜ์ૹͷϑϩϯτΤϯυʹؔ܎͢ΔϚΠΫϩαʔϏεΛDocker Swarm͔Β Kubernetes΁Ҡߦͨ͠࿩Λܝࡌ͍ͯ͠·͢ɻ ͜ͷαʔϏεͰ͢ → https://live.nicovideo.jp/ ϋϯυϒοΫͷ঺հ χίχίੜ์ૹ WebϑϩϯτΤϯυͷKubernetesҠߦϋϯυϒοΫ 2022 GitHub https://github.com/dwango/nicolive-kubernetes-migration-handbook-2022 Web Site https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/ PDF https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/about/
6. ҎԼͷ՝୊Λ࠷খݶʹ͍ͨ͠ • KubernetesΛӡ༻͢Δʹ͸େྔͷ஌͕ࣝඞཁ • ࣮ӡ༻͢Δʹ͸அยతͳ஌ࣝͰ͸ͳ͘Ұ؏ͨ͠৘ใ͕ٻΊΒΕΔ • ઐ໳ੑ͕ߴ͍ͱଐਓԽͷϦεΫ͕ߴ͍ • ֶशίετ͕ߴ͍ʢ= આ໌ʹ΋͕͔͔࣌ؒΔʣ ࣄྫͱͯ͠࢒͢ҙٛ • ઌߦࣄྫ͕͋Δͱଞͷ։ൃऀ͕ํ๏Λਅࣅ͠΍͘͢ͳΔ • ΋ͬͱ͍͍ํ๏Λࢥ͍ͭ͘ਓ͕ग़ͯ͘Δ ϋϯυϒοΫΛެ։ͨ͠໨త
7. ϋϯυϒοΫͰ঺հ͍ͯ͠ΔτϐοΫ ঺հτϐοΫ πʔϧ or ߏ੒ ؆қ·ͱΊ ✅ Manifest؅ཧ TypeScript ܕγεςϜͱ൚༻ݴޠͷΤίγεςϜʹॾʑΛ৐ͤΔ ✅ Continuous Delivery Argo CD + Argo Rollouts + Slack Bot ෳ਺؀ڥରԠ + Canary Releaseɺӡ༻ࣗಈԽ Service Mesh Istio Observability޲্ Rate Limit Local Ratelimit / Global Ratelimit ෛՙରࡦ εέʔϦϯά Horizontal Pod Autoscaler HPAͷݟੵ΋Γํ๏ ✅ ෛՙࢼݧ vegetaʢHTTP Load Testing Toolʣ ҠߦͷͨΊͷ؆қੑೳධՁ ✅ ϞχλϦϯά dd-agent → DataDog rpsͷूܭɺPodͷߋ৽ՄࢹԽ Logging nginx → fluent bit → Volume → fluentd ϩάऩूͷҠߦͱকདྷ΁ͷ෍ੴ ✅ Kubernetes΁ͷҠߦ ετϥϯάϥʔϑΟάύλʔϯ Docker Swarm͔ΒKubernetes΁ͷҠߦ ✅ ͕͍͍ͭͯΔ߲໨ΛຊൃදͰ঺հ͢Δ
8. എܠઆ໌ͱ՝୊
9. എܠ ߟྀࣄ߲ Kubernetes͸ΦϯϓϨ؀ڥʹߏங • ໰୊͕ੜͨ͡৔߹͸VMͷઃఆ΋ߟྀʹೖΕΔඞཁ͕͋Δ • Kubernetesߏங࣌ͷઃఆ΋৔߹ʹΑͬͯ͸ݟΔ Kubernetesͷӡ༻ऀ͸σϓϩΠ͠ ͨνʔϜࣗ਎ • WebϑϩϯτΤϯυͷνʔϜ͕ࣗ෼ͨͪͰσϓϩΠ͍ͯ͠Δίϯςφͷ໘౗ΛݟΔ • Backend For Frontendͷߏ੒ͳͨΊɺ༷ʑͳϚΠΫϩαʔϏεʹ઀ଓ ӡ༻ऀʹे෼ͳKubernetesͷ஌ࣝ ͕ٻΊΒΕΔ • ଞͷνʔϜͱ΋ڠྗ͸͍ͯ͠Δ͕ඞͣ͠΋ඞཁͳͱ͖ʹ͕࣌ؒऔΕΔΘ͚Ͱ͸ͳ͍ • ΞϓϦέʔγϣϯͷ໰୊ͳͷ͔ɺΠϯϑϥͷઃఆͷ໰୊ͳͷ͔ݟۃΊΔඞཁ͕Ͱͯ͘Δ ΠϯϑϥͷGitOpsʹ׳Ε͍ͯͳ͍ • ैདྷ͸Jenkinsܦ༝Ͱߋ৽ɻύϥϝʔλʔΛೖΕͨΒϦϦʔε͞ΕΔੈք • Ҡߦ࣌ͷӡ༻Λ؆୯ʹͯ͠GitOpsʹνʔϜ͕׳ΕΔ·Ͱͷ͕࣌ؒඞཁ ։ൃ؀ڥ؀ڥ͕ෳ਺ • ෳ਺ϨʔϯͰҊ͕݅ಈ͍͍ͯΔ͜ͱ΋͋ΔͨΊૣ͍΋ͷ͕ͪ • ։ൃ؀ڥ΋εέʔϧ͍͖͍ͯͨ͠ എܠઆ໌ (1/2)
10. എܠઆ໌ (2/2) Backend For Frontedͱ͸ nodejs MicroServie A MicroService B MicroService C Browser ᶆ HTML / JSON ᶃ Access ᶄFetch ᶅ ɾServe Side Rendering ɾData Formating Load Balancer • ΫϥΠΞϯτʢWebϑϩϯτΤϯυʣͷͨΊͷαʔόʔ • Server Side RenderingʢHTMLΛฦ٫ʣ • ੔ܗ͞ΕͨσʔλΛฦ͢APIͷఏڙΛߦ͏ʢ෗ഊ๷ࢭ૚ͱͯ͠ͷ໾ׂʣ • جຊతʹεςʔτϨε Reverse Proxy
11. ༧૝͞ΕΔ՝୊ ͳͥ ͲͷύϥϝʔλʔΛ͍ͭߋ৽ͯ͠Α͍ͷ͔Θ͔Βͳ͍ • ӡ༻υΩϡϝϯτ͕ͳ͍ • ύϥϝʔλʔ͕ଟ͍ɾґଘؔ܎͕ෳࡶ ࡉ͔͍ϨϏϡʔ͕ൃੜ͢Δ • YAMLͷه๏͕༷ʑ • ໋໊نଇ / ॱং … etc શମ࠷దԽ͕೉͍͠ • ྔ͕ଟ͍ɾݕূ߲໨͕ଟ͍ • ࠶ݱੑͷ͋Δߏ੒Λҡ࣋ɺల։͢Δํ๏Λ͍࣋ͬͯͳ͍ υΩϡϝϯτΛॻ͍ͯ΋อक͠ͳ͘ͳΔ • ࣮ଶͱဃ཭͍ͯ͘͠ • υΩϡϝϯτΛϨϏϡʔର৅ʹීஈೖΕͯͳ͍ ϞχλϦϯάʹඞཁͳύϥϝʔλʔΛ෇༩͠๨ΕΔ • ීஈ͔Βҙࣝతʹ৮Δ͜ͱ͕গͳ͍ എܠΛ౿·্͑ͨͰ... KubernetesΛΞϓϦέʔγϣϯ։ൃऀ͕ແରࡦͰӡ༻͢Δͱൃੜ͢Δʢͨ͠ʣͰ͋Ζ͏՝୊
12. ՝୊͕ൃੜ͢Δͱ͜Ζ͸େ͖͘2Օॴ ManifestΛॻ͘ͱ͖ͷ໰୊ σϓϩΠ͢Δͱ͖ͷ໰୊ Manifest (YAMLϑΝΠϧ) Developer Kubernetes ฤू Apply
13. YAMLΛੜ੒ ࠓճͷ՝୊ղܾͷΞϓϩʔν Manifest (YAML) σϓϩΠͷ՝୊Λղܾ͢Δ Manifest؅ཧ ਺ͷ՝୊Λղܾ͢Δ ӡ༻ͷ՝୊Λղܾ͢Δ Developer͔ΒݟΔͱ׳Ε͍ͯΔπʔϧͰKubernetesͷઃఆม ߋΛͰ͖Δ͜ͱʹͳΔ Developer Deploy webhook ࡉ͔͍ߏ੒มߋ ೔ৗͷߋ৽࡞ۀ
14. TypeScriptͰ ManifestΛॻ͘
15. DEMO: ManifestΛॻ͍͍ͯΔྫ Reference: Playground: https://codesandbox.io/s/typescript-to-kubernetes-manifest-playground-nco13z?file=/src/manifests/deployment.ts SampleCode: https://gist.github.com/Himenon/ec9ab8681a779daf746903ab3d8cddab
16. TypeScriptΛॻ͘͜ͱͰޮ཰ԽͰ͖Δ࡞ۀ • KubernetesͷTypeScriptܕఆٛ͸OpenAPI͔Βੜ੒͢Δ ◦ Schema: https://github.com/kubernetes/kubernetes/tree/master/api/openapi-spec ◦ ੜ੒ͨ͠TypeScript: https://github.com/Himenon/kubernetes-typescript-openapi ▪ ࣗಈੜ੒ ▪ npmʹpublish • ܕͷิ׬Λޮ͔ͤͳ͕Β֤छύϥϝʔλʔΛೖྗ͢Δ • ManifestͷϓϩύςΟͷҙຯ͕Θ͔Βͳ͍ͱ͖͸TSDocΛಡΉ ◦ OpenAPI Schemaͷdescription͕EditorͰݟΕΔͨΊWebݕࡧ͠ͳͯ͘΋͍͍ • ࣗ෼ͨͪͷϚΠΫϩαʔϏεͷ৘ใΛTSDocΛॻ͘͜ͱ΋Ͱ͖Δ • ม਺Λར༻ͯ͠ύϥϝʔλʔԽ • ࢦఆ͞Εͨ͘ͳ͍஋͸ExceptionΛ౤͛Δ ◦ CIͰݕ஌ͤ͞ΔͨΊɺcatch͠ͳͯ͘Α͍
17. TypeScriptͰManifestΛग़ྗ͢ΔΞʔΩςΫνϟͷղઆ Reference: Handbook: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/manifest/kubernetes-manifest-generator-architecture/ SlackBot͔Βௐ੔͢Δ৔ॴ ؀ڥ͝ͱͷόʔδϣϯ৘ใͳͲɻ ManifestҎ֎ʹ΋υΩϡϝϯτ ΋ग़ྗ͢Δ ܕఆٛ͸OpenAPI͔Βੜ੒͠ ͨ΋ͷΛར༻͢Δ ֤ϚΠΫϩαʔϏε͝ͱͷςϯ ϓϨʔτΛੜ੒͓ͯ͘͠ ςϯϓϨʔτͷ੾Γସ͑΍ࡉ͔ ͍ύϥϝʔλʔΛௐ੔͢Δ ग़ྗ͢ΔϑΝΠϧߏ଄ͷ؅ཧ ༷ʑͳϑΥʔϚοτ ʹग़ྗ͢Δॲཧ ߹੒ͷ৔ॴ Port,Namespace, ଞͷMSͷ৘ใͳ Ͳͷݻఆ஋
18. labelɾannocationͷࣗಈ෇༩ • app: micro-service-aͱ͍͏ܗࣜͰؔ࿈͢Δίϯϙʔ ωϯτʹશͯ෇༩ kubectl get service,deployment,configmap,virtualservice -l app=microservice-a ͕Մೳ • ϚΠΫϩαʔϏεͷόʔδϣϯΛߋ৽͢Δͨͼʹ annotationͷ஋΋มԽ͢ΔͨΊɺDataDogͷϩάͷ෼ ղೳ͕޲্ TypeScriptͰॻ͘͜ͱʹΑͬͯԿΛ࣮ݱ͔ͨ͠ʢ1/2ʣ kube_app_versionͰάϧʔϓԽͯ͠ՄࢹԽͨ͠ྫ ͲͷλΠϛϯάͰόʔδϣϯ͕੾ΓସΘͬͨͷ͔ՄࢹԽ͞Εɺ ໰୊ͷૣظൃݟʹͭͳ͕Δɻ Reference: Handbook: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/performance/monitoring/ DataDogͷλάநग़: https://docs.datadoghq.com/ja/containers/kubernetes/tag/?tab=containerizedagent
19. Reference: Handbook: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/manifest/kubernetes-manifest-generator-architecture/ ߏ੒มߋ͕؆୯͔ͭਝ଎ʹͰ͖Δ • ߏ੒͕ςϯϓϨʔτԽ͞Ε͍ͯΔͨΊɺύϥϝʔλʔ̍ͭͷߋ৽Ͱେ୾ͳߏ੒มߋ͕ՄೳʹͳΔ શମ࠷దԽ͕୹࣌ؒͰՄೳ • શͯͷManifest͸ੜ੒͞ΕΔͨΊ਺ͷ໰୊Ͱ͸ͳ͘ͳͬͨɻ Manifest͸શͯ୯ҰϦϙδτϦ಺ʹ׬શͳঢ়ଶͰग़ྗ͢Δ • Manifest͸׬શͳঢ়ଶͰϦϙδτϦʹอଘ͞ΕΔͨΊࠩ෼͕ͦͷ··ͷҙਤΛද͢ʢม਺͕հࡏ͠ͳ͍ͷͰ೴಺Ͱ૊ΈཱͯΔඞཁ͕ ͳ͍ʣ • GitHub্ͰͷݕࡧʹHit͠΍͘͢ͳΔ ൚༻ݴޠΛͦͷ··ར༻͢ΔͨΊ඼࣭Λҡ࣋͢Δख๏Λ༻ҙ͠΍͍͢ • ໋໊نଇ΍஋ͷ্ݶɺԼݶͷςετͳͲ TypeScriptͰॻ͘͜ͱʹΑͬͯԿΛ࣮ݱ͔ͨ͠ʢ2/2ʣ
20. ArgoCD + Slack Bot
21. ArgoCDͷSyncͷ༏ઌ౓ Parent Apps Appsͷॱং؅ཧ ᶅApps ֤ϚΠΫϩαʔϏε ᶄApps Secretͷ࡞੒ Root ج఺ ᶃNamespace Namespaceͷ࡞੒ App of AppsύλʔϯΛར༻ͯ͠੹຿෼ׂ • ᶃ Namespaceͷ࡞੒ • ᶄ Secret • Secret͸ϚΠΫϩαʔϏεͰར༻͢Δ஋͕֨ೲ͞Ε͍ͯΔ͕ɺ ؅ཧऀΛݶఆ͢ΔͨΊ෼ׂ • ᶅ ֤ϚΠΫϩαʔϏε • NamespaceͱSecret͕લஈ֊Ͱ࡞੒͞Ε͍ͯΔͨΊؾ݉Ͷͳ͘σϓϩΠͰ͖Δ • ͜ͷ഑ԼʹσϓϩΠ͍ͨ͠ΞϓϦέʔγϣϯ͕࿈ͳΔ • ׆ൃʹมߋ͕ߦΘΕΔϚΠΫϩαʔϏε܈ Reference: Handbook: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/ci/argo-cd/ How Do I Configure Waves?: https://argo-cd.readthedocs.io/en/stable/user-guide/sync-waves/#how-do-i-configure-waves
22. PcWeb path: /overlays/[env]/pc-web ArgoCD AppsͷࢀরઌϦϙδτϦ ᶅApps ֤ϚΠΫϩαʔϏε BasicPcWeb path: /overlays/[env]/basic-pc-web BasicSpWeb path: /overlays/[env]/basic-sp-web IstioIngressGateway path: /overlays/[env]/istio-ingress-gateway ϞϊϨϙ಺ͷσΟϨΫτϦߏ੒ • overlays/[σϓϩΠ؀ڥ]/[ϚΠΫϩαʔϏε໊] ※ NamespaceͱSecretͷManifest͸আ͘ Reference: Handbook: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/ci/argo-cd/ Sample Repositroy: https://github.com/Himenon/kubernetes-template
23. @bot update version:2.0.0 app:pc-web • Slack্ͰBotʹMentionΛඈ͹͢ @bot merge pr:123 • ࡞੒͞ΕͨPull RequestΛϚʔδ͢Δ @bot create next release • ຊ൪޲͚ʹ࣍ͷϦϦʔεΛ࡞੒ ◦ ࣍ͷόʔδϣϯͷܾఆʢSemantic Releaseʣ΍ νέοτͷ࡞੒ɺϦϦʔεϊʔτͷ࡞੒ͳͲ ॾʑͷ࡞ۀ΋ؚΉ → ೔ৗͷߋ৽͸git clone͠ͳͯ͘΋ྑ͍ Slack BotʹΑΔόʔδϣϯߋ৽ͷࣗಈԽ Reference: Handbook: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/ci/slack-bot/ Sample Repositroy: https://github.com/Himenon/template-slack-bolt
24. ݕূ࡞ۀ
25. • ૄ௨֬ೝ ◦ ϚΠΫϩαʔϏεؒͷ઀ଓ͕Ͱ͖Δ͔ ◦ Ϋϥελʔͷ಺֎ͱͷૄ௨͕Ͱ͖Δ͔ • ෛՙ଱ੑ ◦ େྔͷΞΫηε͕͋ͬͨ৔߹ʹ௨৴Λҡ࣋Ͱ͖Δ͔ ◦ ௨৴༰ྔͷΩϟύγςΟΛ௒͑ͳ͍͔ • ηΩϡϦςΟ ◦ ඞཁ࠷௿ݶHeader৘ใͷΈΛར༻͢Δ ◦ Ϋϥελʔ֎ʹର͢ΔΞΫηε੍ݶ ͜ΕΒΛ֬ೝ͍ͯͨ͘͠Ί͍ʹ͸ϩΪϯάͱϞχλϦϯάج൫͕ඞཁ • Grafana / Prometheus / DataDogΛར༻ ԿΛݕূ͢Δ͔ Reference: Handbook: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/performance/load-test/
26. ݕূͷਐΊํʢ1/3ʣ γϯϓϧͳHTTPαʔόʔΛ༻ҙͯ͠ຊ൪؀ڥ·ͰઌߦϦϦʔε͢Δ • ඞཁͳݴޠʢgo, nodejs, ruby, python, etc…)Ͱ༻ҙ͢Δͱͳ͓ྑ͍ • ͨͱ͑૯౰ΓͰΞΫηε͞Εͯ΋໰୊͕ͳ͍Α͏ͳαʔϏεΛຊ൪ͷKubernetesʹσϓϩΠ HTTP Server Browser ࣍ͷΑ͏ͳ΋ͷΛ࣮૷͓ͯ͘͠ͱݕূ͕ḿΔ ΫΤϦύϥϝʔλʔ΍HeaderͰઃఆͰ͖Δͱศར • Logग़ྗ • Metricsͷग़ྗ • ٙࣅߴෛՙॲཧ ◦ CPU Intensive ◦ MEM Intensive • ஗ԆϨεϙϯε ◦ setTimeoutͰ3000msec଴ͨͤΔͳͲ • αΠζͷେ͖͍Ϩεϙϯε ◦ ඇѹॖͷը૾΍ڊେͳJSONΛΘ͟ͱฦ͢ͳͲ • HTTP2ͷ௨৴ Reference: Sample Repositroy: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/performance/load-test/#ࢼݧํ๏
27. ݕূͷਐΊํʢ2/3ʣ ݕূ؀ڥͰ͸࿈ଓతʹϦΫΤετΛૹΓଓ͚Δ • ࿈ଓతͳϦιʔεͷมԽ΍ϩά͕ه࿥ͱͯ͠࢒Δ ◦ ։ൃ؀ڥͰ͸ؾܰʹ͋ͪͪ͜Ͱߋ৽͍ͯ͠ΔͨΊɺߋ৽࣌ͷෆ۩߹͕ૣظʹᖰΓग़ͤΔ • ݕূ؀ڥ༻ͷΞϥʔτ΋ਖ਼͘͠ൃใͤ͞Δ • Horizontal Pod Autoscalar͕࣮ߦ͞ΕΔλΠϛϯά͕ഽؒͰ͔ͭΊΔ ◦ → ͜ͷઃఆ͸ةͳ͍ͳͱ͍͏ײ͕֮਎ʹͭ͘ • Kubernetes্ͰNodejsΛಈ͔ͨ͠ͱ͖ͷৼΔ෣͍͕͔ͭΊΔ → ϦΫΤετͷঢ়گ΋։ൃ/ຊ൪ҰகΛ໨ࢦ͢ Reference: Sample Repositroy: https://github.com/Himenon/http-echo X. ։ൃ/ຊ൪Ұக: https://12factor.net/ja/dev-prod-parity
28. ݕূͷਐΊํʢ3/3ʣ ຊ൪؀ڥͰϦΫΤετͷϛϥʔϦϯάΛ࣮ࢪ͢Δ • nginxͷϦΫΤετͷϛϥʔϦϯάػೳΛར༻ • ຊ൪ͱಉ౳ͷϦΫΤετΛड͚෇͚ͯૄ௨ͱෛՙͷ ঢ়ଶͷಈ࡞֬ೝΛ࣮ࢪ → ͜͜·Ͱ΍Ε͹ෛՙͱૄ௨ͷݒ೦͕΄ͱΜͲແ͘ͳΔɻ Reference: Handbook: https://dwango.github.io/nicolive-kubernetes-migration-handbook-2022/docs/performance/load-test/#proxy͔ΒϦΫΤετΛmirroring͢Δ ngx_http_mirror_module: http://nginx.org/en/docs/http/ngx_http_mirror_module.html
29. ݕূΛͯ͠Θ͔Δ͜ͱͷ·ͱΊ ݕূͯ͠Θ͔Δ͜ͱ ݪҼ / ରࡦ ։ൃ/ຊ൪؀ڥͰͷ݁߹֬ೝ ඞཁͳࣄ຿࡞ۀɺࣄલͷઃఆͷߋ৽ɺ࠶ىಈͷ༗ແͳͲ͕Θ͔Δ ೔ৗͷϦϦʔε࡞ۀͰϘτϧωοΫʹͳΔλεΫ Slack BotͰࣗಈԽରԠ Graceful ShutdownΛ࣮૷͍ͯ͠ͳ͍ Retryॲཧͷ͓͔͛ͰͰϢʔβʔӨڹ͸ͳ͍͕ɺμογϡϘʔυ্ͰΤϥʔͱͯ͠؍ଌ͞Ε͍ͯΔ ௨৴ͷϘτϧωοΫ Pod਺͕଍Γͳ͍ɻRate Limit͕؇͍ͳͲ HPA͕ىಈ͢Δ͔Ͳ͏͔ େ఍ͷ৔߹ܭࢉϛε Memory Leak MEMͷ࢖༻ྔͷඍ෼ΛऔΔͱϝϞϦϦʔΫ͍ͯ͠Δ͜ͱΛ͕Θ͔Δɻ 1ϲ݄ޙʹOut of memory͢Δ͕ͦΕ·Ͱʹߋ৽͞Ε͍ͯΔͨΊϦηοτ͞Ε͍ͯͨɻ ΞϓϦέʔγϣϯͷੑೳݶքʢԼݶ/্ݶʣ Nodejs(expressjs)ΛKubernetes্ͰՔಇͤͨ͞৔߹ͷͦ΋ͦ΋ͷ্ݶ͕Θ͔Δɻ ্ݶ = ͋Δͱ͜Ζ͔Βrps͕৳ͼͳ͘ͳΔͱ͜Ζ Nodeͷ࢖༻཰͕ภ͍ͬͯΔ PodAntiAffinityΛར༻ͯ͠PodΛ෼ࢄ഑ஔͤ͞Δ Server Side RenderingͱAPI͕ಉډ͍ͯ͠ΔΑ͏ͳPod͸ޓ͍ʹϦι ʔεΛ৯͍߹ͬͯΔ VirtualServiceΛར༻ͯ͠URI୯ҐͰPodΛ෼ׂ͢ΔΑ͏ʹରԠ ϞχλϦϯάͷ݁Ռ͔ΒϦΫΤετ਺ͱϦιʔε࢖༻ྔͷ૬ؔΛ༧ଌ Ͱ͖Δ େن໛ͳΠϕϯτͰ͸Ͳͷఔ౓ඞཁ͔ɺϛʔςΟϯάͰૈݟੵ΋ΓΛग़ͤΔΑ͏ʹͨ͠
30. ΫϥελʔͷҠߦ
31. Ҡߦͷํ๏ ετϥϯάϥʔ ϑΟά ύλʔϯ Stranglerύλʔϯ • ετϥϯάϥʔ ϑΟά ύλʔϯ - Azure Architecture Center | Microsoft Docs ػೳͷಛఆͷ෦෼Λ৽͍͠ΞϓϦέʔγϣϯ΍αʔϏεʹঃʑʹஔ͖׵͑Δ͜ͱͰɺϨΨγ γεςϜΛஈ֊తʹҠߦ͠·͢ɻ ϨΨγ γεςϜ͔Βͷػೳ͕ஔ͖׵͑ΒΕ͍ͯ͘ͱɺ৽͍͠γεςϜ͸࠷ऴతʹݹ͍γεςϜͷػೳ͢΂ͯΛஔ͖׵͑ɺݹ͍ γεςϜΛ཈ѹͯ͠࢖༻ఀࢭͰ͖ΔΑ͏ʹ͠·͢ɻ ઌߦࣄྫ • ৽نߏங΍Ҡߦ࣌ͷϦεΫΛܰݮɺʮετϥϯάϥʔύλʔϯʯͱ͸ʁɿಛूɿϚΠΫϩαʔϏεೖ໳ʢ5ʣ - ˏIT • EnvoyΛFront Proxyͱͯ͠ར༻͢Δ - Uzabase for Engineers
32. γϯϓϧͳHTTPαʔόʔͱϛϥʔϦϯάͷݕূΛ׬ྃͨ͠ޙ Phase 1: Ҡߦલͷঢ়ଶ Phase 2: Ҡߦதͷঢ়ଶ • IngressGateway͔ΒDocker SwarmʹProxy • IngressGatewayͷෛՙΛ֬ೝ͢Δ Phase 3: Ҡߦޙͷঢ়ଶ Phase 2ɺ3͸ݕূʹΑͬͯݟੵ΋Γ݁Ռ͕͋ΔͨΊɺ͔ͦ͜Βେ ͖ͣ͘Εͳ͍ݶΓ͸PhaseΛ໭͢͜ͱ͸ͳ͍ɻ χίχίੜ์ૹͷBFFαʔόʔ͸̏ͭͷҠߦϑΣʔζͰҠߦ
33. ϚΠΫϩαʔϏεͷҠߦͷखॱ Ҡߦͷॱ൪ • جຊઓུͱͯ͠ϦΫΤετ਺ͷখ͞ͳϚΠΫϩαʔϏε͔Β։࢝͢Δ • ӡ༻ʹඞཁͳσʔλ͕ෆ଍͍ͯ͠Δ৔߹͸Phase 3͔ΒPhase 1΍2·Ͱ໭͢͜ͱ΋͋Δ Ҡߦͷप஌ • εϓϨουγʔτͰͲͷΞϓϦέʔγϣϯΛͲͷ࣌ظʹɺͲͷPhaseʹ͢͢ΊΔ͔ΛνʔϜ಺֎ʹप஌ • มߋ͕͋ͬͨ৔߹͸೔࣌ͷϛʔςΟϯάͰ࠶౓प஌ ※ Ҡߦঢ়ଶʹ໊લΛ͚ͭͯ఻ୡ͢Δ͜ͱͰೝࣝ߹Θͤɺਐḿ؅ཧΛ༰қʹ͍ͯ͠Δɻ ࡾาਐΜͰೋาԼ͕Δײ͡ͰҠߦΛ׬਱ɻ
34. શମ·ͱΊ TypeScriptͰManifestΛ؅ཧ͍ͯ͠Δ • WebϑϩϯτΤϯυͷνʔϜ͕ӡ༻Ͱ͖Δঢ়ଶʹɻυΩϡϝϯτ͸TSDocʹهࡌɻ • ExceptionΛthrow͢Ε͹ͦΕ͕σετʹɻ • ։ൃ࣌͸ؾʹͯ͠ͳ͍ӡ༻ʹඞཁͳύϥϝʔλʔ͸ࣗಈతʹ෇༩͞ΕΔΑ͏ͳ࢓૊Έʹͨ͠ • μΠφϛοΫͳߏ੒มߋ͕؆୯ʹ • ຊ࣭తͳ࡞ۀʹूதͰ͖Δ ArgoCD + Slack Bot • ArgoCD͸৽نͷΫϥελʔ্ཱͪ͛௚࣌͠ʹށ࿭Θͳ͍Syncͷઃܭʹ • SlackBotʹΑͬͯҠߦதɺҠߦޙͷӡ༻࣌ͷෛՙΛԼ͛Δ ݕূ࡞ۀ • ϝτϦΫεͷऩूͱՄࢹԽ • γϯϓϧͳHTTPαʔόʔΛຊ൪ϦϦʔεͯ݁͠߹֬ೝ • ։ൃ؀ڥͰ͸ϦΫΤετΛૹΓଓ͚ɺϩάɾϝτϦΫεɾߋ৽࣌ͷॠஅͷ৘ใͳͲΛऩू͠ରࡦʹճ͢ Ҡߦ࡞ۀ • ετϥϯάϥʔ ϑΟά ύλʔϯͰ࣮ࢪ • ϞχλϦϯάͭͭ͠૝ఆ಺ͷڍಈΛ͍ͯ͠Ε͹ϦϦʔε • νʔϜ಺ͷप஌