Share
Embed
Wasmで広がるEnvoyとIstioの世界
Kubernetes Meetup Tokyo #40 https://k8sjp.connpass.com/event/206303/
mathetake
March 25, 2021
Transcript
67 slides
-
• Takeshi Yoneda (Ϛελέ) / Twitter, Github: @mathetake • Software
Engineer at Tetrate.io • “Paid” OSS dev: Envoy, Istio, Proxy-Wasm, TinyGo • C++ committer of Proxy-Wasm project • Creator of Go SDK for Proxy-Wasm • Contributor of V8 whoami -
1. Introduction to WebAssembly 2. Background: Envoy’s extensibility 3. Proxy-Wasm:
WebAssembly For Proxies 4. Proxy-Wasm in Istio 5. Challenges and Future Agenda -
• Stack-basedͳԾϚγϯͱͦͷ༷ • ݩʑϒϥβ(JS)ͷߴԽ͕త • asm.js -> WebAssembly(Wasm)ͱਐԽ • ༷ΛಡΊ͔Δ͕Ұݴ
“host” ͷཁٻ͕ͳ͍ • Portable, Platform-agnostic, Open-ended • Run at near-native speed: ΊͬͪΌ͍(※࣮ʹΑΔ) • Security: ελοΫ͕ϓϩάϥϜ͔Βݟ͑ͳ͍ͱ͔ͦ͏͍͏ͷ WebAssembly 101 -
• ༷ʑͳݴޠ͔ΒίϯύΠϧՄೳ: C, C++, Rust, Go(TinyGo), AssemblyScript • ॳjsͷΈࠐΈ͕લఏ: ͦΕͧΕͷݴޠ͕ಠࣗͷ
“glue.js”Λ࣋ͭ • ίϯύΠϥڞ௨ͷ“Platform”λʔήοτ͕ͳ͍(͍, jsͳΜ͚ͩͲ͞, Έ͍ͨͳ) • VMͱͯ͠༏ल&ίϯύΠϥج൫͋Δͷʹମͳ͍ • ϒϥβͷ֎Ͱ͍͍ͨ WebAssembly 101 -
• Wasm <-> Host OSͷ͠ํ(ΠϯλʔϑΣΠε)=SystemcallΛඪ४Խ͠·͠ΐ͏ • WasmͷίϯύΠϥͷ“platform”λʔήοτʹ͠Α͏ • WASI (WebAssembly
System Interface)ͷొ WebAssembly 101 https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/ -
• WASIΛ࣮ͨ͠ϥϯλΠϜ͕ొ • WAVM, Wasmtime, Wasmer, Lucet, ʑ • “ී௨ͷϓϩηε”ͱಉ͡Α͏ʹWasm͕ಈ͘Α͏ʹͳΔ
• KrustletͳͲͷ৺తϓϩδΣΫτͷొ Wasm gets out of web browsers -
• WASI = Wasmͷ༷ʹ͋Δ “host function” Λsystem call͚ʹඪ४Խͨ͠ͷ • ݁ہABI
(Wasm <-> Hostͷ͠ํ)ܾ͑͞ΊΕͳΜͰͰ͖Δ • ҙͷΞϓϦͷதͰVMΛಈָ͔͍ͯ͜͠͠ͱ͕Ͱ͖ͦ͏ʂ • WASIʹݶΒ༷ͣʑͳABI͕ొ • Blockchain༻ͷABI • k8s֦ு༻ͷABI • Proxyαʔό༻ͷABI(Proxy-Wasm) Wasm gets out of web browsers WASI ABI Linux / Darwin / Windows / … Wasm Virtual Machine -
• WASI = Wasmͷ༷ʹ͋Δ “host function” Λsystem call͚ʹඪ४Խͨ͠ͷ • ݁ہABI
(Wasm <-> Hostͷ͠ํ)ܾ͑͞ΊΕͳΜͰͰ͖Δ • ҙͷΞϓϦͷதͰVMΛಈָ͔͍ͯ͜͠͠ͱ͕Ͱ͖ͦ͏ʂ • WASIʹݶΒ༷ͣʑͳABI͕ొ • Blockchain༻ͷABI • k8s֦ு༻ͷABI • Proxyαʔό༻ͷABI(Proxy-Wasm) Wasm gets out of web browsers Proxy-Wasm ABI Envoy / Nginx / ATS / … Wasm Virtual Machine -
• “Cloud-native high-performance edge/middle/service proxy” • CNCF Graduated Project, Github
Star: 16,000+ • Written in C++ • αʔϏεϝογϡͷData planeͱͯ͠།Ұແೋͷଘࡏ What is Envoy? -
• ϓϩΩγαʔόʔͳͷͰuse case͕ແݶ • EnvoyʹવϓϥάΠϯ(֦ு)ػߏ͕ଘࡏ • ֦ுͷྫ • ࣾಠࣗͷAuthn/Authz ػೳΛೖΕ͍ͨ
• ಛघͳϓϩτίϧΛѻ͍͍ͨ • MySQL, Redis, DynamoDB, etc. Envoy’s extensibility -
• Envoy੩తϦϯΫ͕લఏ • C++Ͱॻ͔ͳ͍ͱ͍͚ͳ͍ • มߋөͷͨΊʹશ࠶ىಈ͕ඞཁ • StableͳABI͕ܾ·͍ͬͯͳ͍ • ֦ுػೳΛՃ͢Δʹ
1. UpstreamϚʔδ͢Δ —> ಛघ(private)ͳϢʔεέʔεෆՄ 2. ಠࣗϏϧυΛ͢Δ —> upstreamͷै͕େม(෦ͷAPI͕unstable) Envoy’s extensibility -
• ղܾ͍ͨ͠՝ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυඞཁͳ͍ • Security-awareͳ༷
Envoy’s extensibility -
• ղܾ͍ͨ͠՝ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυඞཁͳ͍ • Security-awareͳ༷
Envoy’s extensibility -
• ղܾ͍ͨ͠՝ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυඞཁͳ͍ • Security-awareͳ༷
Envoy’s extensibility -
• ղܾ͍ͨ͠՝ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυඞཁͳ͍ • Security-awareͳ༷
Envoy’s extensibility -
• ղܾ͍ͨ͠՝ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυඞཁͳ͍ • Security-awareͳ༷
Envoy’s extensibility -
• ղܾ͍ͨ͠՝ҎԼͷ௨Γ • ಈతʹϓϥάΠϯͷload͕Ͱ͖Δ • ෳݴޠͷαϙʔτ • EnvoyࣗମͷಠࣗϏϧυඞཁͳ͍ • Security-awareͳ༷
Envoy’s extensibility -
• ϓϥάΠϯ = WasmͷόΠφϦͱͯ͠Envoyʹ • Envoyͷ“தͰ” WasmͷVMΛಈ͔͢ • Envoy <->
Wasmͷ͠ํ(ABI)Λstableʹ͠Α͏ • ͯ͢ͷ՝ΛΫϦΞ • ҙͷݴޠ, ಈతload, ηΩϡΞ, stableͳABI Extending Envoy with Wasm ಠࣗͷABI -
• ͦͦϓϩΩγαʔόͷ֦ு༻ABIීวతͳͷͳͣ • ྫ: Http Header/Body/Trailerʹରͯ͠◦◦͢Δ, tcpίωΫγϣϯʹରͯ͠xx͢Δ • Envoy͔ΒΓͤΔͷͰʁ =>
Proxy-Wasmͱ͍͏Envoy͔Βಠཱͨ͠ϓϩδΣΫτʹ Extending Envoy with Wasm Proxy-Wasm -
• ̐ͭͷݴޠͷSDK͕͋Γ, ؆୯ʹcompatibleͳWasmͷίϯύΠϧ͕Մೳ • C++, Rust, Go, AssemblyScript • Hostͷެ࣮ࣜC++ͷΈ
• https://github.com/proxy-wasm/proxy-wasm-cpp-host • VMͱͯ͠ Wasmtime, V8, WAVM͕ݱঢ়ར༻Մೳ • Envoy, Apache Traffic Server(PoC)͜ͷެ࣮ࣜΛϥΠϒϥϦͱͯ͠͏ Proxy-Wasm: WebAssembly for Proxies -
Proxy-Wasm Spec: Example 2. http request header౸ண࣌, ϗετ͔ΒݺΕΔ “Wasm”ͷؔ http
request body͕౸ண࣌, ϗετ͔ΒݺΕΔ “Wasm”ͷؔ -
• 1 VM / (Plugin, Worker Thread) • Nativeͷ֦ுͷதͰproxy-wasm- cpp-hostΛ༻
• 1ͷؒ “envoy-wasm”ͱ͍͏ forkઌͰ։ൃ • 2020/10ʹupstreammerge • v1.17ͰॳͷstableϦϦʔε Proxy-Wasm in Envoy: Impl. model -
• ࣮͜ΕΒͷϓϥάΠϯਖ਼֬ʹσϑΥϧτͰ“Wasm”Ͱಈ͍͍ͯͳ͍ • Proxy-WasmͷABIΛ༻͍͍ͯΔ͕NativeʹCompile & Envoyʹstatic link • NullVmͱզʑ͕ݺΜͰ͍ΔΈ •
EnvoyͷWasm Extension͕·ͩalphaεςʔλεͳͨΊ • ҰํͰɺWasmͱͯ͠ίϯύΠϧՄೳͳΑ͏ʹίʔυ͕ॻ͔Ε͍ͯΔ • ͍ۙকདྷ(v1.10~)σϑΥϧτͰWasm VMͰϓϥάΠϯ͕ಈ͘Α͏ʹͳΔͣ Istio: official plugins in Proxy-Wasm -
• ݱঢ়Wasm plugin༻ͷ API(CRD)ଘࡏ͠ͳ͍ • EnvoyFilterͰؤுΔ͔͠ͳ͍ • Wasm-SIGͰ1st classͳ APIΛࡦఆத
How to deploy plugins in Istio: API -
• ݱঢ়Istio (Istio-agent), http(s)Ͱserving͞ΕͨWasmόΠφϦͷΈLoadՄ • OCI-imageͱͯ͠WasmͷbinaryΛOCI registryʹ֨ೲ͢ΔྲྀΕ͕͋Δ • https://github.com/engineerd/wasm-to-oci •
https://github.com/solo-io/wasm-image-spec • Proxy-Wasm༻ͷOCI-image specΛIstio Wasm-SIGͰࡦఆத: ΄΅ݻ·ͬͨ • ͍ۙকདྷOCI registryʹpush͞ΕͨWasmΛIstio͕αϙʔτ͢ΔΑ͏ʹͳΔ • (࣮ͷTODOͰ͢…) How to deploy plugins in Istio: Delivery -
Future of Workflow of Istio Wasm plugins Build Push OCI-registries
k apply -f wasm.yaml Image: hoge.com/my-plugin:v1.10 config: … -
• ABI͕Envoyͷ࣮ʹ͍ͩͿد͍ͬͯΔ(ͨΓલͱ͍͑ͨΓલ…) • 1VM͕ෳͷϦΫΤετΛࡹ͘ͱ͍͏ઃܭ • V8 / WAVMͷoverhead͕എܠ • 1VM
= 1 requestͱ͍͏Ϟσϧͷ߹ʹෆཁͳઃఆ͕͍͔ͭ͋͘Δ Challenges in Proxy-Wasm -
• ςετͱσόά͕͔ͳΓਏ͍ • ςετͷͨΊʹEnvoyΛಈ͔͢…? • Go SDKͰEnvoyͷΤϛϡϨʔλΛGoͰॻ͍ͯ, nativeίϯύΠϧ Ͱςετ: ݶք͕͋Δ
• WasmଆʹdebugͷͨΊͷ༷͕ͳ͍ͷͰ, ֤ݴޠ͝ͱʹ৭ʑ • LLVMϕʔεͷݴޠͳΒDWARF͕όΠφϦʹೖͬͯΔͷͰ৭ʑͰ͖ Δ͕Proxy-Wasm HostଆͰະ࣮(Ջ͕͋ͬͨΒΔ) • Stack trace͕औΓͮΒ͍(Ϣʔβʔۭ͔ؒΒstackݟΕͳ͍ͷͰ) Challenges in Proxy-Wasm -
Future of Cloud Native Wasm 2015 2019/03 2020/03 WASIͷొ WASMͷొ
Proxy-Wasmͷొ 2013/03 asm.jsͷొ 2020/04 Krustletͷొ 202x/yy ???? -
• WasmηΩϡΞͰportableͳόΠφϦϑΥʔϚοτ+ԾϚγϯͷ༷ • CNCF͕༧ଌ͢ΔΑ͏ʹ, ༷ʑͳ໘Ͱར༻͕૿͖͑ͯͦ͏ • Cloud Native Wasm DayͷτʔΫΛݟΔͱಈ͕ݟ͑ͯ͘Δ?
• Proxy-WasmʹΑΓEnvoy/Istio͕WasmͰ֦ுͰ͖ΔΑ͏ͳੈք͕طʹདྷ͍ͯΔ • Isito͚ͩͰͳ͘, େاۀ͕طʹproductionͰಈ͔͍ͯ͠Δ • ·ͩ·ͩൃల్্Ͱ͕͢, 2021தʹ͔ͳΓख़͢ΔݟࠐΈ ·ͱΊ