Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Adventure around Kubernetes at Chatwork
Search
Ryo Sakamoto
January 23, 2020
Technology
6
7.9k
Adventure around Kubernetes at Chatwork
Ryo Sakamoto
January 23, 2020
Tweet
Share
More Decks by Ryo Sakamoto
See All by Ryo Sakamoto
いろいろなAWSアカウントのArgo CDを統合した話
cwsakamoto
1
1k
ArgoCDとGitHub Self Hosted Runnerを使って リリース時間を1/4にした話
cwsakamoto
0
2.3k
チャットワークにおけるKubernetesOnAWS.pdf
cwsakamoto
0
91
チャットワークにおけるKubernetesOnAWS.pdf
cwsakamoto
0
84
Kubernetes on AWS at Chatwork
cwsakamoto
0
1.7k
Other Decks in Technology
See All in Technology
ClaudeCodeにキレない技術
gtnao
1
930
Microsoft Fabric ガバナンス設計の一歩目を考える
ryomaru0825
1
160
LIXIL基幹システム刷新に立ち向かう技術的アプローチについて
tsukuha
1
920
PHPからはじめるコンピュータアーキテクチャ / From Scripts to Silicon: A Journey Through the Layers of Computing
tomzoh
2
350
M365アカウント侵害時の初動対応
lhazy
0
700
[SRE NEXT 2025] すみずみまで暖かく照らすあなたの太陽でありたい
carnappopper
2
710
組織内、組織間の資産保護に必要なアイデンティティ基盤と関連技術の最新動向
fujie
0
440
How Do I Contact Jetblue Airlines® Reservation Number: Fast Support Guide
thejetblueairhelpsupport
0
210
Frontier Airlines Customer®️ USA Contact Numbers: Complete 2025 Support Guide
frontierairlineswithflyagent
0
100
データ戦略部門 紹介資料
sansan33
PRO
1
3.3k
Semantic Machine Intelligence for Vision, Language, and Actions
keio_smilab
PRO
2
340
無理しない AI 活用サービス / #jazug
koudaiii
0
110
Featured
See All Featured
YesSQL, Process and Tooling at Scale
rocio
173
14k
Fireside Chat
paigeccino
37
3.5k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Building Applications with DynamoDB
mza
95
6.5k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.3k
Building Adaptive Systems
keathley
43
2.7k
Facilitating Awesome Meetings
lara
54
6.5k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
357
30k
Building an army of robots
kneath
306
45k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Transcript
,VCFSOFUFTΛΊ͙Δݥ +"846(ίϯςφࢧ෦ $IBUXPSL43&νʔϜࡔຊ
© Chatwork ࣗݾհ 2 ▸ ࡔຊ ྒ (͔͞ͱɹΓΐ͏) ▸ Chatwork
SREΤϯδχΞ ▸ ϥϯχϯάΤϯδχΞ ▸ ϑϧϚϥιϯ͕Α͏͘3࣌ؒ17͙Β͍(20193݄) ▸ 2019ͷߦڑ2700km͙Β͍ ▸ NY- LA4000kmͳͷͰ, ೦ͳ͕ΒΞϝϦΧԣஅͰ͖ͣ
▸ ϏδωενϟοταʔϏε ▸ άϧʔϓνϟοτɺλεΫཧɺ ϑΝΠϧڞ༗ɺϏσΦɾԻ௨ ▸ ಋೖاۀ 246,000 ࣾ (201912݄ݱࡏ)
© Chatwork ΞδΣϯμ 4 ▸ ChatworkͷKubernetesͷྺ࢙ΛৼΓฦΓͳ͕Βݱࡏͷӡ༻ʹ͍ͭͯ ▸ ྺ࢙ ▸ ӡ༻πʔϧ
▸ ChatworkͷKubernetes CI/CD
© Chatwork ҙ 5 Fargateͷ͋Γ·ͤΜ
© Chatwork લఏ$IBUXPSLͷ,VCFSOFUFTར༻ܗଶ 6 ▸ Ϛϧνςφϯτ ▸ ΫϥελࣗମSRE͕࡞Δ ▸ ཧܥΞϓϦέʔγϣϯ(Datadog,
Fluentd ͳͲ) -> SRE ▸ αʔϏεΞϓϦέʔγϣϯ -> DevνʔϜ
© Chatwork $IBUXPSLͷ,VCFSOFUFTͷྺ࢙ 7 ▸ ಋೖ2016 ▸ ࠷ॳͷProductionͷόʔδϣϯ1.5 ▸ ϝοηʔδϯάγεςϜͷ৽ϓϩδΣΫτ
▸ AWS Summit Tokyo 2017 Ͱൃද ▸ ͦͷޙԿ͔ΫϥελҠߦ(όʔδϣϯΞοϓ)Λͭͭ͠ɺݱࡏʹࢸΔ ▸ ࣗલKubernetes, EKSͷࠞ߹ঢ়ଶ
© Chatwork &,4ͷಋೖ 8 ▸ EKSར༻લkube-awsͱ͍͏ͷΛར༻ͯ͠, ࣗલͰϗεςΟϯά ▸ https://github.com/kubernetes-incubator/kube-aws ▸
EKSͷొʹΑΓkube-awsͷϢʔβݮ… ▸ ࡉ͔͍ػೳʹόά͕ଟ͘, దٓPRରԠͭͭࣗ͠લύονӡ༻ ▸ EKSࣗମίϯτϩʔϥʔ෦͚ͩͰ, ͍ʹ͍͘ͳ͊…ͱϓϨϏϡʔͰ ࢥ͍͕ͬͯͨeksctlʹΑΓͦ͏͍͏ͱ͜Ζ͕·Δͬͱָʹ
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 9 ▸ eksctl ▸ ։ൃ͕ૣ͍ ▸ جຊతʹΫϥελͷߏupdateͰ͖ͳ͍
▸ ϊʔυάϧʔϓͷઃఆมߋΛ͍ͨ͠߹ɺ৽ن࡞ɾچϊʔυά ϧʔϓআ ▸ ϊʔυάϧʔϓͷߏΛࢼ͍ͯ͠Δͱ͖ͪΐͬͱखؒ
ΦϑΟγϟϧͳͷͰɺEKS৽͍͠ػೳେମAWSͷൃදͱಉ࣌ʹରԠ͞Ε͍ͯΔ
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 11 ▸ kube-aws ▸ ΫϥελͷߏupdateՄೳ ▸ ϊʔυάϧʔϓͷઃఆมߋΛ͍ͨ͠߹ɺچϊʔυάϧʔϓ͔Β
ϩʔϦϯάΞοϓσʔτ(CloudFormationͷupdateͳͷͰ...) ▸ EKSΛར༻͢ΔͷͰͳ͘, controller, etcdؚΊͯ࡞ ▸ ੜKubernetesͳͷͰɺΧελϚΠζੑ͔ͳΓߴ͍ ▸ KubernetesͷઃఆϑΝΠϧ͕େͳyamlʹͳΔ(Productionͷ686ߦ)
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 12 ▸ ظతʹݟΔͱ, eksctlͷ΄͏͕ϝϦοτେ͖͍ ▸ Ϣʔβೝূ͕iamͰͰ͖ͨΓ, podʹiam
roleΛ͚ΒΕͨΓ, eksͦͷͷ ͷϝϦοτେ͖͍ ▸ iam role for podͱͯخ͍͠ ▸ kube2iam, kiam, kube-aws-iam-controllerͯ͢ӡ༻͖ͯͨ͠ܦݧ ▸ ͜Ε͜ΕͰखܰ͋͞Δ͚ΕͲ
ΫϥελΛ࡞Δ͚ͩͳΒ͜ΕͰOK ͔͠͠ɾɾ
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 14 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜ΕΛΓӽ͑Δӡ༻ɾΈ͕ඞཁ
© Chatwork ,VCFSOFUFTͷӡ༻Λࢧ͑Δπʔϧͨͪ 16 ▸ eksctl (kube-aws) ▸ Ϋϥελࣗମ͜ΕͰ࡞ ▸
Variant ▸ https://github.com/mumoshu/variant ▸ helm, helmfile ▸ https://github.com/roboll/helmfile
© Chatwork 7BSJBOU 17 ▸ task runner ▸ eksctlͳͲͷίϚϯυΛϥοϓ͢Δ ▸
wrapper shellͰ͍͍͕ɺͪ͜ΒyamlͰॻ͚ͯɺύϥϝʔλͳͲͷࢦ ఆΓ͍͢ ▸ dockerϕʔεͷtask runner͕ఆٛͰ͖ɺڥґଘΛݮΒͤΔ ▸ ڥͱίϚϯυΛηοτʹͰ͖Δ
© Chatwork FLTDUM͚ͩͰͳͥͩΊͳͷ͔ 18 ▸ eksctl͚ͩͰेͳ߹ଟ͍ ▸ ͱΓ͋͑ͣࢼݧڥΛىಈͤ͞ΔʹίϚϯυ͚ͩͰे ▸ ͨͩ͠ࡉ͔͍ઃఆΛ͠Α͏ͱ͢Δͱ,
ϑΝΠϧͰͷઃఆͷ΄͏ָ͕ ▸ sampleҎ֎ͷυΩϡϝϯτͳ͍ͷͰ, ҙ ▸ https://github.com/weaveworks/eksctl/blob/master/pkg/apis/eksctl.io/ v1alpha5/types.go ͔Β୳͔͢͠ͳ͍ͱ͖͋Δ ▸ ઃఆΛͤΔVariantͰϥοϓͯ͠, eksctl༻ઃఆϑΝΠϧͷΛஔ
# Ϋϥελ໊ # αϒωοτ # cluster-autoscalerͷiam roleͭ͘Δ # eksຊମ(ϊʔυάϧʔϓআ͘)ͷeksctlͷઃఆϑΝΠϧ
© Chatwork FLTDUM͚ͩͰͳͥͩΊͳͷ͔ 20 ▸ Ϋϥελ࡞Δ͚ͩͳΒ, ୯ମར༻Ͱ͍͚ͳ͘ͳ͍ ▸ FluxͩΕ͕ೖΕΔͷɺͷରԠ ▸
Flux ... GitOps tool ޙड़͠·͢ ▸ GitOpsͰઃఆͷөΞϓϦέʔγϣϯͷσϓϩΠΛ͢Δ ▸ Ͱ, GitOpsΛ࣮ݱ͢ΔͨΊͷπʔϧҰମ୭͕ೖΕΔͷ͔...! ▸ GitOps Kubernetes෦ʹ࣮ݱ͢ΔͨΊͷΞϓϦέʔγϣϯ͕͍Δ
© Chatwork 21 ▸ eksctlίϚϯυͰFluxΛ͍ΕΔ͜ͱՄೳ ▸ υΩϡϝϯτϕʔεͰରԠͰ͖Δ͕ɺԼهͷϑϩʔΛ·ͱΊ͍ͨ ▸ eksctlͷઃఆϑΝΠϧ࡞ ▸
eksctl create cluster ▸ eksctl enable repo ... (Fluxͷinstall) ▸ ͜ΕΒΛvariantͰλεΫ੍ޚ(Infrastructure as codeͷҰ) FLTDUM͚ͩͰͳͥͩΊͳͷ͔
© Chatwork 22 FLTDUMΛ7BSJBOUͰXSBQͨ݁͠Ռ ͜Ε͚ͩͰEKSڥOK https://en.wikipedia.org/wiki/Achlis EKS࡞ίϚϯυΛachlisͱ໋໊
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 23 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜͜·ͰKubernetesࣗମͷߏங ͔͜͜ΒKubernetes෦ͷߏங
© Chatwork 25 ▸ https://toris.io/2019/12/what-i-think-about-when-i-think-about-kubernetes- and-ecs/ ▸ (ͳΜ͔ଜ্य़थͷλΠτϧͬΆ͍...) ▸ EKSϚωʔδυͰ͕͢ɺϚωʔδϝϯτίϯιʔϧͰ࡞͚ͨͩ͠Ͱ
΄΅ԿͰ͖·ͤΜ ▸ ϊʔυάϧʔϓ(࣮ࡍʹίϯςφ͕ಈ͘ڥ)Λ࡞ͬͯɺ·ͩϩάૹΕ ͣɺϝτϦΫεݟΕͣɺͰ͢ ▸ ͳͷͰɺEKSͰཧܥͷΞϓϦέʔγϣϯ(ex. Datadog)ͷӡ༻͕ඞਢ &$4ͱͷҧ͍
© Chatwork IFMN IFMNpMF 26 ▸ helm ▸ The package
manager for Kubernetes ▸ KubernetesʹΞϓϦέʔγϣϯΛೖΕΔࡍʹɺඞཁͳͷΛύοέʔδ Խͯ͠ɺೖΕͯ͘ΕΔπʔϧ ▸ ChatworkͰ΄ͱΜͲΞϓϦέʔγϣϯΛHelmΛར༻ͯ͠σϓϩΠ ▸ Chartࣗମڥʹґଘ͠ͳ͍ʂ ▸ σϓϩΠͷ࠶ݱੑ(ࣗલKubernetes(kube-aws) -> EKSͰͦͷ··)
© Chatwork IFMN IFMNpMF 27 ▸ helmfile ▸ helm chartΛͲ͏ͬͯద༻͢Δͷ͔Λએݴతʹॻ͚Δ
▸ helmͰσϓϩΠ͢ΔࡍͷΞϓϦέʔγϣϯͷઃఆɺhelmίϚϯυ ͷΦϓγϣϯͳͲΛҰׅͰهࡌ͠ɺhelmͰσϓϩΠͰ͖Δπʔϧ ▸ https://github.com/helm/charts/tree/master/stable ▸ ͍ΖΜͳΞϓϦέʔγϣϯͷhelm chart ▸ https://developers.freee.co.jp/entry/2019/12/03/122657
© Chatwork ͳͥIFMNpMF͕ඞཁͳͷ͔ 28 ▸ ΞϓϦέʔγϣϯ͝ͱʹ͍͍ͨΦϓγϣϯ(લॲཧ)͕มΘΔ ▸ ྫ͑)ΞϓϦέʔγϣϯಉ࢜Ͱґଘؔͷ͋Δͷ, ͳͲ ▸
helmͷઃఆϑΝΠϧ͕ڥ͝ͱ(test, stg, prod)ͰมΘΔ ▸ ྫ͑)AWSͷΞΧϯτ͝ͱʹҧ͏, ϩάͷు͖ग़͠ઌ͕ҧ͏ ▸ go template ͑ΔͷͰ, helmͷઃఆΛಈతʹੜ͢Δ͜ͱՄೳ ▸ ChatworkͰɺ͞ΒʹVariantͰhelmfileΛแΈࠐΈ
© Chatwork 29 IFMNpMF IFMN Λ7BSJBOUͰXSBQͨ݁͠Ռ ͜Ε͚ͩͰKubernetes෦OK
© Chatwork 30 ͍Ζ͍Ζ7BSJBOUͰXSBQͨ݁͠Ռ
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 31 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜͜·ͰKubernetesӡ༻ ͔͜͜ΒCI/CD
© Chatwork ,VCFSOFUFTͷӡ༻ͰҰ൪େมͳ͜ͱ 33 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
© Chatwork ,VCFSOFUFTڥͷ$*$% 34 ▸ KubernetesͰͷCI/CD ▸ αʔϏεͷΞϓϦέʔγϣϯ ▸ Concourse
CIΛར༻ͯ͠CI/CD ▸ ͯ͢ΛyamlܗࣜͷpipelineͰఆٛͰ͖Δπʔϧ
© Chatwork ,VCFSOFUFTڥͷ$*$% 35 ▸ ࠔͬͨ ▸ CIͳΓϩʔΧϧ͔ΒσϓϩΠ͢Δ߹, ςϯϓϨʔτԽ͍ͯ͠Δͱͯ͠ ɺΫϥελ͕Ͱ͖Δͨͼʹ͖ઌΛม͑Δඞཁ͕͋Δ
▸ ͦͦαʔϏεΞϓϦͷσϓϩΠΛߦ͏Concourse CIࣗΛͲ͜Ͱӡ ༻͢Δͷ͔
© Chatwork (JU0QTʂ 36 https://www.weave.works/technologies/gitops/
© Chatwork (JU0QTʂ!$IBUXPSL 37 Argoͷ෦·ͩରԠͰ͖͍ͯͳ͍
© Chatwork $IBUXPSLͷ(JU0QTߏ 38 ▸ Flux + Argo CDͷϋΠϒϦουߏ ▸
Flux ▸ ѻ͑ΔͷݪଇmanifestͷΈͰ, 1flux - 1repo - 1branch ▸ Argo CDࣗͷΠϯετʔϧnamespace, aws-auth(eksಠࣗ)ͳͲ manifestద༻Ͱ͋·Γมߋ͕ͳ͍ͷ ▸ Argo CD ▸ GUI͕͋ͬͨΓ, helmʹରԠ͍ͯͨ͠Γɺͱػೳ͕ଟ͍͍͘͢
© Chatwork $IBUXPSLͷ(JU0QTߏ 39 ▸ αʔϏεͷΞϓϦέʔγϣϯ͚ͩͰͳ͘ɺཧܥͷΞϓϦͷΞϓϦέʔ γϣϯGitOpsͰద༻͢Δํࣜʹมߋ༧ఆ ▸ Kubernetes͕Ͱ͖ΔͱɺඞཁͳΞϓϦέʔγϣϯͯ͢উखʹೖ͍ͬͯ Δঢ়ଶ
▸ αʔϏεͷΞϓϦέʔγϣϯʹؔͯͪ͠ΖΜΓସ͑ͳͲ͕ඞཁ
© Chatwork (JU0QTΛΊ͙Δڥ 40
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 41 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
© Chatwork ·ͱΊ 42 ▸ Chatworkʹ͓͚ΔKubernetesͷӡ༻(ߏங) ▸ eksctl, variant, helm,
helmfile ▸ GitOpsߏ ▸ EKSɺֹʂ