Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
自分だけの仮想クラスタを高速かつ効率的に作る kubefork
Search
Kazuki Obata
February 25, 2025
Technology
0
160
自分だけの仮想クラスタを高速かつ効率的に作る kubefork
Wantedly Tech Night 〜ウォンテッドリーで内製している開発用ツールの紹介〜
https://wantedly.connpass.com/event/341010/
Kazuki Obata
February 25, 2025
Tweet
Share
More Decks by Kazuki Obata
See All by Kazuki Obata
散らばったトレースを繋げる技術
donkomura
1
640
ウォンテッドリーのインフラチームに加わってみて
donkomura
0
150
AWS CLI で気軽にコスト改善やってみた
donkomura
1
160
入門 KRR
donkomura
0
250
Other Decks in Technology
See All in Technology
C++26アップデート 2025-03
faithandbrave
0
1.2k
AIとSREで「今」できること
honmarkhunt
3
690
2025-04-14 Data & Analytics 井戸端会議 Multi tenant log platform with Iceberg
kamijin_fanta
0
180
AI 코딩 에이전트 더 똑똑하게 쓰기
nacyot
0
460
Goの組織でバックエンドTypeScriptを採用してどうだったか / How was adopting backend TypeScript in a Golang company
kaminashi
12
9.1k
SnowflakeとDatabricks両方でRAGを構築してみた
kameitomohiro
1
570
読んで学ぶ Amplify Gen2 / Amplify と CDK の関係を紐解く #jawsug_tokyo
tacck
PRO
1
300
Linuxのパッケージ管理とアップデート基礎知識
go_nishimoto
1
700
Notion x ポストモーテムで広げる組織の学び / Notion x Postmortem
isaoshimizu
1
150
Pythonデータ分析実践試験 出題傾向や学習のポイントとテクニカルハイライト
terapyon
1
110
MySQL Indexes and Histograms – How they really speed up your queries
lefred
0
150
Aspire をカスタマイズしよう & Aspire 9.2
nenonaninu
0
360
Featured
See All Featured
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
How to train your dragon (web standard)
notwaldorf
91
6k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.4k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
How to Ace a Technical Interview
jacobian
276
23k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
19
1.2k
BBQ
matthewcrist
88
9.6k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
41
2.3k
Statistics for Hackers
jakevdp
798
220k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Adopting Sorbet at Scale
ufuk
76
9.3k
Transcript
© 2025 Wantedly, Inc. ͚ࣗͩͷԾΫϥελΛߴ͔ͭ ޮతʹ࡞Δ kubefork Wantedly Tech Night
#8 2025-02-25 Kazuki Obata
© 2025 Wantedly, Inc. ࣗݾհ ڊി थ (Obata Kazuki) @donkomura_
Wantedly, Inc. (2024/09 ~ ) ΠϯϑϥΤϯδχΞ #k8s #ϘϧμϦϯά🧗
© 2025 Wantedly, Inc. ࣍ • Ϟνϕʔγϣϯ • ՝ͱΓ͍ͨ͜ͱ •
Ͳ͏࣮ͬͯݱ͢Δ͔ ◦ Wantedly ։ൃͷ߹ ◦ ΞΠσΟΞͱ • ·ͱΊ
© 2025 Wantedly, Inc. Ϟνϕʔγϣϯɿ։ൃதͷϓϨϏϡʔ
© 2025 Wantedly, Inc. Ϟνϕʔγϣϯɿ։ൃதͷϓϨϏϡʔ • ؾܰʹϓϨϏϡʔͰ͖Δͱศར ◦ ຊ൪ͱಉ͡Α͏ͳڥͰಈ࡞֬ೝ͠ͳ͕Β։ൃͰ͖Δ ◦
ϨϏϡʔ͍͢͠ Πϝʔδɿhttps://github.com/marketplace/actions/deploy-pr-preview ։ൃதʹ ࣅͨΑ͏ͳ͜ ͱ͕͍ͨ͠
© 2025 Wantedly, Inc. ཧɿݱঢ়ͷ։ൃϓϩηεͷτϨʔυΦϑ ͍͍ͱ͜औΓΛ͍ͨ͠ ݸਓͷ։ൃαʔόʔʢओʹϩʔΧϧʣ - ✅ڥ͕ଞͷӨڹʹΑͬͯԚ͞Εͳ͍ -
✅Docker ͳͲͰຊ൪ʹ͍ۙڥ࣮ݱͰ͖Δ - ❌Πϯϑϥ෦ͯ͢Λ࠶ݱͰ͖ΔΘ͚Ͱ ͳ͍ - ❌ґଘͨ͠αʔϏε͕͋Δͱ։ൃ͕͍͠ ڞ༗ͷ։ൃαʔόʔʢओʹϦϞʔτʣ - ✅ຊ൪ʹ͍ۙڥͰ։ൃͰ͖Δ - ❌ଞͷਓͷ࡞ۀ͔ΒӨڹΛड͚Δɾ༩͑Δ - ❌҆ఆͨ͠ڥΛอͪʹ͍͘ - յΕΔ͜ͱ͕͋Δ - ❌ͷಛఆ͕͘͠ͳΔ - σϓϩΠ͕িಥ͢Δ - ϩά͕ࠞࡏ͢Δ
© 2025 Wantedly, Inc. Ͳ͏࣮ݱ͢Δ͔ - Wantedly ։ൃͷ߹
© 2025 Wantedly, Inc. Ͳ͏࣮ݱ͢ΔɿWantedly ։ൃͷ߹ લఏ • ϚΠΫϩαʔϏεʢΞʔΩςΫνϟʣ •
Πϯϑϥ Kubernetes ◦ 1Ϋϥελ60ݸͷϚΠΫϩαʔϏεΛӡ༻ • ։ൃऀ50ਓ
© 2025 Wantedly, Inc. ࣮ݱͷͨΊͷΞΠσΟΞɿΫϥελʔؙ͝ͱίϐʔ͢Δ • શһʹಉ͡։ൃڥΛఏڙ͢Δͷ͕ཧ • ͯ͢ͷϦιʔεΛຊ൪͔Βίϐʔ͢Δͱ࣮ݱͰ͖Δ ◦
1αʔϏε1αʔόʔͱͯ͠60 x 50 = 3000 αʔόʔ࡞͢Δ͜ͱʹͳΔ 🤔 ◦ Ϧιʔε͕५ʹ͋ΔΘ͚Ͱͳ͍
© 2025 Wantedly, Inc. ۪ʹ࣮ݱ͢Δͷͦ͠͏
© 2025 Wantedly, Inc. ͢Δ
© 2025 Wantedly, Inc. มߋ͍ͨ͠෦͚ͩઐ༻ ΓͷαʔϏεڞ༻
© 2025 Wantedly, Inc. • deployment ͱ service ͚ͩมߋ͢Δ
• ϧʔςΟϯά ◦ ԾΫϥελͷΞΫηεใΛ HTTP Header, gRPC Metadata Ͱൖ
© 2025 Wantedly, Inc. deployment ͱ service ͚ͩมߋ͢Δ
© 2025 Wantedly, Inc. ʮมߋ͍ͨ͠෦͚ͩʯઐ༻ • ΞϓϦέʔγϣϯ։ൃऀʹΑΔมߋ ◦ Stateless ͳϦιʔε͚ͩΛίϐʔ͢Εे
◦ Kubernetes ͩͱ Deployment • Deployment ͷίϐʔʹىҼ͢Δมߋ ◦ ίϐʔͨ͠ Deployment ʹ͚ͯϦΫΤετΛྲྀ͍ͨ͠ ◦ Kubernetes Ͱ Service ΛͬͯτϥϑΟοΫΛసૹ͢Δ
© 2025 Wantedly, Inc. ʮมߋ͍ͨ͠෦͚ͩʯઐ༻ • deployment ͱ service ͷΈίϐʔ͢Δ
◦ มߋ͍ͨ͠ϚΠΫϩαʔϏεͷ Deployment ͱ Service Λίϐʔ ◦ มߋ͠ͳ͍ϚΠΫϩαʔϏεڞ༻ͷͷΛ͏Α͏ʹϧʔςΟϯά
© 2025 Wantedly, Inc. ϧʔςΟϯά
© 2025 Wantedly, Inc. • มߋ෦ʢService, DeploymentʣͷΈΛίϐʔ ◦ ԾΫϥελʹΞΫηε͢Δ߹ ▪
ଞͷϚΠΫϩαʔϏε͔ΒͷϦΫΤετΛ͜ΕΒʹྲྀ͍ͨ͠ ▪ ϓϨϏϡʔͰ͖Δঢ়ଶʹ͍ͨ͠ Ͳ͏ͬͯίϐʔͨ͠ Service, Deployment ʹ ϦΫΤετΛྲྀ͔͢ ͜͜·Ͱ A B B
© 2025 Wantedly, Inc. Istio ͷϦιʔεΛͬͨϧʔςΟϯά • VirtualService ◦ ϚΠΫϩαʔϏεͷαʔϏεؒ௨৴ͷϧʔςΟϯάΛઃఆ͢Δ
resource ◦ header ͷΛجʹϦΫΤετͷϧʔςΟϯάઌΛมߋ͢Δ apiVersion: networking.istio.io/v1beta1 kind: VirtualService spec: hosts: - wantedly-x http: - match: - headers: x-fork-identifer: exact: donkomura route: - destination: host: kube-fork-donkomura x-fork-identifier ͱ͍͏ header Λݟͯ ࣗઐ༻ͷ host ʹ ϧʔςΟϯά͢Δ ίϐʔࡁΈͷ host
© 2025 Wantedly, Inc. ࣮Ͱ͖ͨ ӡ༻Ͱ͖Δʁ
© 2025 Wantedly, Inc. fork ʹ͓͚Δ VirtualService ӡ༻ͷ՝ • 1ͭͷ
VirtualService ʹԾΫϥελ͝ͱͷઃఆ͕ඞཁ • ྫ: 2ਓ͕ಉ͡αʔϏεΛมߋ͍ͨ͠ͱ͖ ◦ Ұͭͷ Virtual Service ʹ݅Λ·ͱΊΔ ◦ x-fork-identifer: cluster-A ʹϚον͢ΔͳΒ ▪ service-A ϧʔςΟϯά ◦ x-fork-identifer: cluster-B ʹϚον͢ΔͳΒ ▪ service-B ϧʔςΟϯ ◦ ͦΕҎ֎ͳΒ service-X ϧʔςΟϯά
© 2025 Wantedly, Inc. fork ʹ͓͚Δ VirtualService ӡ༻ͷ՝ • 1ͭͷ
VirtualService ʹԾΫϥελ͝ͱͷઃఆ͕ඞཁ • ྫ: 2ਓ͕ಉ͡αʔϏεΛมߋ͍ͨ͠ͱ͖ ◦ Ұͭͷ Virtual Service ʹ݅Λ·ͱΊΔ ◦ x-fork-identifer: cluster-A ʹϚον͢ΔͳΒ ▪ service-A ϧʔςΟϯά ◦ x-fork-identifer: cluster-B ʹϚον͢ΔͳΒ ▪ service-B ϧʔςΟϯ ◦ ͦΕҎ֎ͳΒ service-X ϧʔςΟϯά ධՁॱΛ੍ޚͰ͖ͳ͍ ίϯϑϦΫτ͢Δ
© 2025 Wantedly, Inc. VSConfig • VSConfig ◦ ԾΫϥελͷϧʔςΟϯάઃఆΛهड़͢ΔͨΊͷಠࣗϦιʔε ◦
࣮ΧελϜίϯτϩʔϥʔ ▪ ઃఆΛूͯ͠ VirtualService Λੜ͢Δ ◦ VirtualService ͷઃఆΛෳਓ͕৮Βͳͯ͘ྑ͍ ▪ ίϯϑϦΫτ͕ى͖ͳ͍ ◦ յΕͯࣗಈͰݩʹΔ VirtualService ͷҟͳΔมߋΛू͢Δ apiVersion: vsconfig.k8s.wantedly.com/v1beta1 kind: VSConfig spec: headerName: x-fork-identifer headerValue: donkomura host: # ϧʔςΟϯάରͷ Service wantedly-x service: # ϦΫΤετΛྲྀ͢ઌͷ Service kube-fork-donkomura
© 2025 Wantedly, Inc. ։ൃதʹԾΫϥελʹϦΫΤετΛඈ͍ͨ͠ • ͜͜·Ͱ ◦ ಛఆͷ header
ͷ͍ͭͨϦΫΤετԾΫϥελʹϧʔςΟϯά͞ΕΔ ◦ ϧʔςΟϯά VirtualService ͰߦΘΕ͍ͯΔ ◦ VirtualService ͷݸʑͷϧʔςΟϯάઃఆ VSConfig ʹهड़͞Ε͍ͯΔ ͜ΕͰे͔ʁ
© 2025 Wantedly, Inc. ϚΠΫϩαʔϏεʹ͓͚ΔϦΫΤετॲཧͷྲྀΕ A B C E ௨ৗͷϦΫΤετ
req B E A B C E req ؒతʹϦΫΤετΛड͚ͯ ڍಈ͕มΘΔ ίϐʔͨ͠ Deployment ԾΫϥελͷ ϦΫΤετ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req ԾΫϥελͷ ϦΫΤετ A B C E ௨ৗͷϦΫΤετ req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req A B C E req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ ԾΫϥελͷ ϦΫΤετ ௨ৗͷϦΫΤετ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req A B C E req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ ԾΫϥελͷ ϦΫΤετ ௨ৗͷϦΫΤετ ʮԾΫϥελͷΞΫηεͰ͋Δʯ ͱ͍͏ใʢίϯςΩετʣΛ ޙଓͷϚΠΫϩαʔϏε ൖͤ͞Δඞཁ͕͋Δ
© 2025 Wantedly, Inc. ԾΫϥελͷϦΫΤετͰ͋Δ͜ͱΛൖ͢Δ • ͜͜·Ͱʢ࠶ܝʣ ◦ ಛఆͷ header
ͷ͍ͭͨϦΫΤετԾΫϥελʹϧʔςΟϯά͞ΕΔ ◦ ϧʔςΟϯά VirtualService ͰߦΘΕ͍ͯΔ ◦ VirtualService ͷཧ VSConfig ͕ߦ͍ͬͯΔ ԾΫϥελͷ ID Λ࣋ͭ header Λ͚ͭͯ ϚΠΫϩαʔϏεͰ͏ɾൖͤ͞Δ → ϦΫΤετ͕ίϐʔͨ͠ Deployment → ԾతͳΫϥελʹΞΫηε͢Δମݧ
© 2025 Wantedly, Inc. ୭͕ header Λ͚͍ͭͯΔʁ Ͳ͏ͬͯ header Λൖ͍ͯ͠Δʁ
© 2025 Wantedly, Inc. ୭͕ header Λ͚͍ͭͯΔʁ Ͳ͏ͬͯ header Λൖ͍ͯ͠Δʁ
ຊͪ͜Β͚ͩઆ໌͠·͢
© 2025 Wantedly, Inc. ʮԾΫϥελʹΞΫηε͍ͯ͠Δʯ͜ͱΛൖ͢Δ
© 2025 Wantedly, Inc. • ಛఆͷ ID ͷ͍ͨϦΫΤετ (e.g. header)
Λड͚Δ ◦ ड͚ͨϚΠΫϩαʔϏεผͷϚΠΫϩαʔϏεϦΫΤετ ◦ ͦͷࡍʹಉ͡ ID Λ͚ͭͯϦΫΤετΛൃߦ͢Δ ԾΫϥελͷIDΛൖ͢Δ A B C E req id: hoge E req id: hoge
© 2025 Wantedly, Inc. • ಛఆͷ ID ͷ͍ͨϦΫΤετ (e.g. header)
Λड͚Δ ◦ ड͚ͨϚΠΫϩαʔϏεผͷϚΠΫϩαʔϏεϦΫΤετ ◦ ͦͷࡍʹಉ͡ ID Λ͚ͭͯϦΫΤετΛൃߦ͢Δ ԾΫϥελͷIDΛൖ͢Δ req id: hoge req id: hoge req id: hoge header Λൖ͞ ͓ͤͯ͘ A B C E req id: hoge E
© 2025 Wantedly, Inc. Context propagation • ʮԾΫϥελAʹདྷͨϦΫΤετͰ͋Δʯͱ͍͏ίϯςΩετΛ ϚΠΫϩαʔϏεʹൖ (propagate)
͢Δ ◦ ྫ ▪ X-Fork-Identifer: fork-a ͷΑ͏ͳ header ͖Ͱ request ͷॲཧத ▪ ผͷ microservice ʹ௨৴Λߦ͏߹ඞͣ X-Fork-Identifer: fork-a Λ͚Δ • servicex ͱ͍͏ϚΠΫϩαʔϏεڞ௨ϥΠϒϥϦͰ࣮ݱ ◦ wrap ͨ͠ΫϥΠΞϯτΛ͏ ◦ ଞͷϦΫΤετ࣌ʹಉ͡ϔομɾϝλσʔλΛ͏ ◦ ͏গ͠Γ͍ͨ > Wantedly Engineering Handbook
© 2025 Wantedly, Inc. ·ͱΊ • kubefork ԾΫϥελΛ࡞͢ΔΈ ◦ ։ൃऀ͕ࣗઐ༻ͷΫϥελΛ͍࣋ͬͯΔ͔ͷΑ͏ͳମݧΛఏڙ͢Δ
• ΠϯϑϥͱΞϓϦέʔγϣϯͷٕज़ΛΈ߹Θͤͯ Ͱ͖͍ͯΔ ◦ Kubernetes (VirtualService, VSConfig etc.), Istio ◦ servicex