Share
Embed
Transcript
30 slides
-
ࣗݾհ Takahashi Kenta ΧγΣגࣜձࣾ όοΫΤϯυΤϯδχΞ ࠓ͢͜ͱ ҎԼͷΑ͏ͳ՝ΛɺxDSΛͬͯղܾ͍ͯ͠ΔΑɺͱ͍͏Λ͠·͢ - ϒϥϯν͝ͱʹڥΛͭͬͯ͘ಈ࡞֬ೝΛ͍ͨ͠ -
αʔϏεΛ·͍ͨͰϒϥϯν͝ͱͷڥͰಈ࡞֬ೝ͍ͨ͠ -
લఏ ΧγΣͷΞʔΩςΫνϟ • Everything runs on Cloud Run • Everything
runs as an API (gRPC) • API Gateway = Envoy (ԣஅతؔ৺ࣄ) Mobile App API Gateway (Envoy) grpc-gateway Service A Service B + Pub/Sub / Cloud Tasks / Scheduler Envoy ͕୲͏ԣஅతؔ৺ࣄ: JWT ೝূ / RBAC / Transcoding TLS / CDN / HTTP Header ૢ࡞ (Wasm) -
Phase 1 ࠷ॳ͏·͍͍ͬͯͨ͘ ϒϥϯν͝ͱʹ Cloud Run ΛσϓϩΠ͠ Mobile ͔ΒϒϥϯνڥʹϦΫΤετ Mobile
App branch ڥ Service A (branch) Service B (main) γϯϓϧͰͳ͠ ୯ҰͷαʔϏεΛมߋ͢Δ͚ͩͷ PR Ͱ ͜ΕͰेʹݕূͰ͖͍ͯͨ -
Phase 2 αʔϏεΛލ͍Ͱͳ͓ϒϥϯνڥΛར༻͍ͨ͠ Mobile → Service A → Service B
(branch) ʹϦΫΤετΛ͍͛ͨ Service A ͷ͖ઌΛຖճखಈͰมߋʁ → ։ൃऀɾϒϥϯν͕૿͑Δ΄Ͳݱ࣮తͰͳ͍ Mobile App Service A B (main) B (br-1) B (br-2) खಈͰ͖ઌΛมߋ͢ΔͷෆՄೳ -
Phase 3 Job API ͰϒϥϯνڥΛར༻͍ͨ͠ ΧγΣͰ Pub/Sub ͔Βݺͼग़͞ΕΔ Job API
gRPC Ͱ࣮͞Ε͍ͯΔ Pub/Sub → Job API (gRPC) ͱ͍͏ྲྀΕ ͜ͷ gRPC ݺͼग़͠ϒϥϯν୯ҐͰ੍ޚ͍ͨ͠ Pub/Sub + Tasks / Scheduler Job API (main) Job API (branch) gRPC ? Job API gRPC → xDS ͰղܾͰ͖Δͣ -
ٻΊΒΕͨཁ݅ • αʔϏεΛލ͍ͰϒϥϯνڥʹϦΫΤετΛ͚Δ • Pub/Sub TasksɺScheduler ͔ΒͷϦΫΤετ ϒϥϯνڥʹ͚Δ →
xDS ϕʔεͷαʔϏεσΟεΧόϦʔͰղܾ -
xDS ͱ ͱͱ Envoy ͷͨΊʹઃܭ͞ΕͨɺϧʔςΟ ϯάѼઌใͳͲͷଓؔ࿈ઃఆΛಈతʹ ৴͢ΔͨΊͷ API ܈ɻ "x"
Listener / Route / Cluster / Endpoint ͳ Ͳ༷ʑͳϦιʔεΛࢦ͠ɺͦΕΒͷ Discovery Service Ͱ xDS grpc-go xDS ΛωΠςΟϒαϙʔτ͓ͯ͠Γ ࠶ىಈͳ͠ʹϧʔςΟϯάΛϦΞϧλΠϜߋ৽Ͱ͖Δ Control Plane (Go / go-control-plane) xDS Data Plane (grpc-gateway / grpc-go) -
xDS ϓϩτίϧͷߏ LDS Listener Discovery ͲͷϙʔτͰ listen ͢Δ͔ RDS Route
Discovery ͲͷϧʔτͰϚονͤ͞Δ͔ CDS Cluster Discovery ͲͷΫϥελʹৼΓ͚Δ͔ EDS Endpoint Discovery ۩ମతͳΤϯυϙΠϯτԿ͔ ϒϥϯν QA Ͱͷ׆༻: ϔομʔͷʹԠͯ͡ ϒϥϯνڥͷ Cloud Run ʹ ಈతʹϧʔςΟϯάΛΓସ͑Δ → LDS ͱ CDS Λ׆༻ -
Proxyless gRPC ͷ xDS ωΠςΟϒαϙʔτ Envoy ͳͲͷαΠυΧʔϓϩΩγΛܦ༝ͤͣ gRPC ΫϥΠΞϯτࣗମ͕ xDS
Λղऍ grpc-go xds:/// εΩʔϜͰ Control Plane ͔ΒϧʔςΟϯάใΛऔಘ͠ ࣗࣗͰϩʔυόϥϯγϯάɾϧʔςΟϯάΛߦ͏ αΠυΧʔෆཁ = ӡ༻ίετݮ ैདྷ (Sidecar) App → Envoy → upstream Sidecar ཧ͕ඞཁ Proxyless (ࠓճͷํࣜ) App + grpc-go (xDS) → upstream Sidecar ෆཁ grpc-go v1.40+ Ͱ xDS αϙʔτ͕҆ఆ -
Why xDS? ΧγΣʹϑΟοτͨ͠ཧ༝ • grpc-go ͕ xDS ΛωΠςΟϒαϙʔτ طଘͷ gRPC
αʔϏεʹͦͷ··ΈࠐΊΔɻՃϓϩΩγෆཁ • Dynamic Configurations αʔϏεͷ࠶ىಈͳ͠ʹϧʔςΟϯάΛϦΞϧλΠϜߋ৽ • Go Ͱ Control-Plane ͷ࣮͕͍͢͠ Go Ͱ Control Plane Λ࣮ɻ https://github.com/envoyproxy/go-control-plane Λͬͯ؆୯ʹ࣮͕Ͱ͖Δ -
OSS cloud-run-service-router-xds ΧγΣ͕։ൃɾެ։ͨ͠ xDS Control Plane Cloud Run ͷϦϏδϣϯใΛݩʹ xDS
Snapshot Λࣗಈੜ͢Δαʔό • go-control-plane ϕʔεͷ Control Plane • Cloud Run Admin API ͰϦϏδϣϯࢹ • ϒϥϯν → Snapshot ͷࣗಈϚοϐϯά • seed.yaml ͰαʔϏεఆٛΛએݴతཧ kauche/ cloud-run-service-router-xds Go 98.9% ˒ 4 v0.0.6 github.com/kauche/cloud-run-service-router-xds -
શମΞʔΩςΫνϟ 1. ϒϥϯνڥͷσϓϩΠ Cloud Run αʔϏε໊: svc-a-main svc-a-feat-payment svc-b-main svc-b-feat-payment
{prefix}-{branch} 2. Control Plane ͕αʔϏεҰཡΛऔಘ cloud-run-service-router-xds Cloud Run Admin API Λఆظతʹݺͼग़͠ σϓϩΠࡁΈαʔϏεɾϦϏδϣϯΛݕग़ ϒϥϯν໊Λநग़ͯ͠ xDS Snapshot Λੜ Cloud Run API 3. xDS ͰϧʔςΟϯάใΛ৴ Control Plane (xDS Server) xDS stream grpc-gateway Service A Service B Service C 4. x-branch ϔομʔʹج͍ͮͯϧʔςΟϯά Mobile (x-branch: feat/payment) → grpc-gateway → Service A (branch) → Service B (branch) -
౷Ұݪଇ: ಛघϔομʔʹΑΔϧʔςΟϯά x-branch: feature/xxx (Ծ໊Ͱ͢) ͜ͷϔομʔ͕͍͍ͯΕ → xDS ͕ϒϥϯνڥʹϧʔςΟϯά ϔομʔ͕ͳ͚Ε
→ main ʹϑΥʔϧόοΫ ͯ͢ͷϒϥϯνϧʔςΟϯά͕͜ͷݪଇʹ౷Ұ͞ΕΔ • ಉظ௨৴ (gRPC): ϔομʔΛαʔϏεؒͰൖ • ඇಉظ௨৴ (Pub/Sub): attributes → ϔομʔʹมͯ͠߹ྲྀ -
ಉظ௨৴ gRPC ͷέʔε Mobile App x-branch: feat/xxx grpc-gateway ← xDS
ϧʔςΟϯά Service A (branch) ϔομʔൖ → Service B (branch) ← grpc-go xDS खಈͷ͖ઌมߋ͕ෆཁɻϔομʔ͕ࣗಈൖ͞ΕΔ -
࣮ grpc-go Ͱͷ xDS ଓ xds:/// εΩʔϜ + Bootstrap ઃఆͰ
xDS ϧʔςΟϯά͕༗ޮʹ // xDS resolver Λ import _ "google.golang.org/grpc/xds" // ଓઌΛ xds:/// εΩʔϜͰࢦఆ conn, err := grpc.Dial( "xds:///service-a.example.com:443", grpc.WithTransportCredentials(...), ) ඞཁͳઃఆ: • GRPC_XDS_BOOTSTRAP ڥมͰ bootstrap ઃఆΛೖ • ଓઌ URI Λ xds:/// εΩʔϜʹมߋ • xds ύοέʔδΛ import (resolver / balancer ͷొ) → grpc-go ͕ Control Plane ͱ௨৴͠ɺϧʔςΟϯάใΛࣗಈऔಘ -
ඇಉظ௨৴ Pub/Sub → Job API (gRPC) ͷέʔε Pub/Sub attr: branch=feat/
xxx attr → x-branch ϔομʔʹม (api-gateway) Job API (gRPC) + xDS ϧʔςΟϯά gRPC ͳͷͰಉظ௨৴ͱಉ͡Έ ϙΠϯτ: Pub/Sub (+ Tasks / Scheduler) Job API Λݺͼग़͢ Job API gRPC Ͱ࣮͞Ε͍ͯΔͨΊ Pub/Sub ͷ attributes Λ x-branch ϔομʔʹม͢Ε ಉ͡ xDS ϧʔςΟϯάʹΔ -
ઃܭͷΩϞ ಉظඇಉظ ʮಛघϔομʔ ʯ ͱ͍͏୯ҰͷϧʔςΟϯάػߏʹू ಉظ: gRPC ϔομʔൖ ඇಉظ: attr
→ header ม xDS ϧʔςΟϯά -
࣮ xDS Control Plane go-control-plane ϥΠϒϥϦͰ Go ࣮ 1 ϒϥϯνͷ
Cloud Run σϓϩΠ 2 Control Plane ͕ݕ 3 xDS Snapshot ߋ৽ 4 grpc-go ͕ ࣗಈө Cloud Run ͷσϓϩΠใΛࢹ͠ɺSnapshot Λಈతʹੜ grpc-gateway / grpc-go gRPC stream ͰมߋΛड৴ → ଈ࣌ө https://github.com/envoyproxy/go-control-plane -
࣮ xDS Bootstrap ઃఆ grpc-go ͕ Control Plane ʹଓ͢ΔͨΊͷઃఆ {
"xds_servers": [{ "server_uri": "xds-server:18000", "channel_creds": [ {"type": "google_default"} ], "server_features": ["xds_v3"] }], "node": { "id": "service-a" } } ઃఆͷϙΠϯτ: server_uri xDS Control Plane ͷΞυϨε channel_creds TLS ೝূใͷઃఆ node.id Control Plane ͕ઃఆΛৼΓ͚Δ ͨΊͷϊʔυࣝผࢠ GRPC_XDS_BOOTSTRAP ڥมͰϑΝΠϧύεΛࢦఆͯ͠ೖ -
࣮ Control Plane ͷ TLS ଓ grpc-go-xds-tls-credentials Control Plane ͷଓʹඞཁͳ
TLS ΫϨσϯγϟϧΛఏڙ͢Δ Go ϥΠϒϥϦ (OSS) ͳͥඞཁ͔: Cloud Run ্ͷαʔϏε͕ Control Plane ʹ҆શʹଓ͢Δʹ Google Cloud ͷೝূใΛͬͨ TLS ͕ඞཁ ఏڙ͢Δػೳ: • google_default credentials ͷࣗಈऔಘ • Bootstrap ͷ channel_creds ͱ࿈ܞ • Cloud Run ͷαʔϏεΞΧϯτೝূ github.com/kauche/grpc-go-xds-tls-credentials -
։ൃڥ Cloud Run API Emulator ϩʔΧϧ։ൃͰ xDS ͷಈ࡞ݕূΛՄೳʹ͢ΔΤϛϡϨʔλ cloud-run-service-router-xds Cloud
Run Admin API ʹґଘ → ϩʔΧϧͰಈ͔͢ʹΤϛϡϨʔλ͕ඞཁ $ docker run --publish 8000:8000 ghcr.io/kauche/cloud-run-api-emulator:0.0.3 docker compose ͰҰൃىಈ: compose.yaml ʹ Control Plane + ΤϛϡϨʔλ + ֤αʔϏεΛఆٛ → ϩʔΧϧͰϒϥϯνϧʔςΟϯάͷ E2E ςετ͕Մೳ github.com/kauche/cloud-run-api-emulator -
࣮ ϔομʔൖͷΈ x-branch ϔομʔ͕ͲͷΑ͏ʹൖ͞ΕΔ͔ Mobile App x-branch: feature/payment Λ༩ grpc-gateway
xDS Ͱ Service A (branch) ʹϧʔςΟϯάɻϔομʔಁա Service A (branch) gRPC metadata ͔Β x-branch Λऔಘ͠ɺoutgoing context ʹൖ Service B (branch) grpc-go xDS ͕ x-branch ʹج͖ͮ branch endpoint ʹଓ gRPC metadata Λͬͨϔομʔൖ Go ͷ grpc.UnaryInterceptor Ͱڞ௨Խ -
Before / After Before • खಈͰ͖ઌมߋ͕ඞཁ • αʔϏεؒ௨৴ͷςετෆՄ • ඇಉظ௨৴ͷϒϥϯνςετෆՄ
• QA ͷฒߦݕূ͕Ͱ͖ͳ͍ • ։ൃऀ͕ΠϯϑϥΛҙࣝ͢Δඞཁ After • ϔομʔ༩͚ͩͰࣗಈϧʔςΟϯά • αʔϏεؒ௨৴ࣗಈै • Pub/Sub ϒϥϯνڥʹ౸ୡ • ෳϒϥϯνΛಉ࣌ݕূ • Platform Team ͕ج൫ఏڙ -
OSS ެ։͍ͯ͠Δπʔϧ܈ kauche/cloud-run-service-router-xds xDS Control Plane ຊମɻCloud Run ϦϏδϣϯΛࢹ͠ Snapshot
Λੜ ˒ 4 kauche/grpc-go-xds-tls-credentials Control Plane ଓ༻ͷ TLS ΫϨσϯγϟϧϥΠϒϥϦ ˒ 4 kauche/cloud-run-api-emulator ϩʔΧϧ։ൃ༻ͷ Cloud Run Admin API ΤϛϡϨʔλ ˒ 12 ͯ͢ github.com/kauche Ͱެ։த -
·ͱΊ 1. طଘͷ grpc-gateway / grpc-go ʹ xDS ΛΈࠐΉ͜ͱͰ Πϯϑϥͷ࠶ߏஙͳ͠ʹϒϥϯνผ
QA ڥΛ࣮ݱ 2. ಉظɾඇಉظΛʮಛघϔομʔ + xDSʯʹ౷Ұ ୯ҰͷϧʔςΟϯάػߏͰ֦ுੑΛ֬อ 3. Go (go-control-plane) Ͱ Control Plane Λ࣮ طଘͷٕज़ελοΫͱͷࣗવͳ౷߹ 4. OSS ͱͯ͠ެ։ → ݟΛίϛϡχςΟʹؐݩ github.com/kauche