Self-Hosted WebAssembly Runtime for Runtime-Neutral Checkpoint/Restore in Edge–Cloud Continuum

the 3rd International Workshop on Middleware for the Computing Continuum (Mid4CC) 2025
https://mid4cc.netsons.org/2025/index.html

Paper: https://dl.acm.org/doi/abs/10.1145/3774898.3778040

Abstract:
Checkpoint/restore (C/R) in the Edge-Cloud Continuum, where edge servers and cloud servers are integrated to combine their respective computational characteristics, enables application live migration for load balancing and service continuity that adapts to user mobility. WebAssembly (Wasm) provides architecture neutrality, allowing bytecode execution across diverse platforms. Furthermore, by leveraging runtime implementations specialized for particular environments and performance optimizations such as Just-In-Time (JIT) and Ahead-Of-Time (AOT) compilation, Wasm bytecode can be executed efficiently by using the runtime best suited to each environment. However, the diversity of runtimes and performance optimizations results in heterogeneous representations of application execution states, making it difficult to apply C/R across multiple runtimes. To address this heterogeneity, we propose a C/R method that employs a self-hosted Wasm runtime compiled into Wasm itself. By checkpointing and restoring application execution states within the self-hosted Wasm runtime, the representation of execution states can be unified. In addition, to mitigate the overhead of duplicated runtime executions, we also investigate optimization techniques for Wasm runtimes designed to be self-hosted. Our evaluation results demonstrate that C/R with a self-hosted Wasm runtime can eliminate runtime differences and enable efficient live migration with a minimal execution state.

Transcript

1. December 15, 2025 3rd International Workshop on Middleware for the Computing Continuum (Mid4CC, 26th ACM/IFIP International Middleware Conference Co-located) Self-Hosted WebAssembly Runtime for Runtime-Neutral Checkpoint/Restore in Edge–Cloud Continuum Yuki Nakata（SAKURA internet Inc. / Future University Hakodate） Katsuya Matsubara（Future University Hakodate）
2. Edge–Cloud Continuum: Distributed Platform by Hetero. Servers Task offloading and handoff are key factors for the continuum Major Collaboration patterns Offloading tasks Edge-to-Edge tasks handoff High performance Low-latency 2
3. Requirements of Platform for Edge–Cloud Continuum 1. CPU Architecture neutrally • Different CPUs may be used at the edge and cloud 2. Efficient Execution • Optimized application execution for different server characteristics • Execution states E.g., hardware performance and constraints 3. Portability • Offloading and handoffs require live migration • Transfers an application and a snapshot by checkpoint/restore (C/R) 3
4. WebAssembly（Wasm): Best Fit Edge–Cloud Continuum Platform Run apps efficiently and anywhere using suitable runtime for each platform Virtual Instruction Set Architecture Compile to Wasm bytecode 4
5. WebAssembly（Wasm): Best Fit Edge–Cloud Continuum Platform Run apps efficiently and anywhere using suitable runtime for each platform Virtual Instruction Set Architecture 100+ Runtimes Tailored for Edge/Cloud* Bytecode Wasmtime Bytecode Low memory footprint WAMR (WebAssembly Micro Runtime) *Yixuan Zhang, Mugeng Liu, Haoyu Wang, Yun Ma, Gang Huang, and Xuanzhe Liu. 2025. Research on WebAssembly Runtimes: A Survey. ACM Trans. Softw. Eng. Methodol. Just Accepted (January 2025). https://doi.org/10.1145/3714465 5
6. WebAssembly（Wasm): Best Fit Edge–Cloud Continuum Platform Run apps efficiently and anywhere using suitable runtime for each platform Virtual Instruction Set Architecture 100+ Runtimes Tailored for Edge/Cloud* Bytecode Wasmtime Bytecode WAMR (WebAssembly Micro Runtime) High performance with JIT compilation *Yixuan Zhang, Mugeng Liu, Haoyu Wang, Yun Ma, Gang Huang, and Xuanzhe Liu. 2025. Research on WebAssembly Runtimes: A Survey. ACM Trans. Softw. Eng. Methodol. Just Accepted (January 2025). https://doi.org/10.1145/3714465 6
7. How to Implement C/R for Wasm Snapshot Need VM states created by runtimes Int a = 1 + 2; Args: … 2 RetAddr: … 1 Locals:… … … Value Stack Frame Stack Stacks 7
8. How to Implement C/R for Wasm Snapshot Need VM states created by runtimes Int a = 1 + 2; Args: … 2 RetAddr: … 1 H E L L O \n Locals:… … … Value Stack Frame Stack Stacks 0x00 0x05 Store non-num values (e.g., array and string) Linear Memory 8
9. How to Implement C/R for Wasm Snapshot Need VM states created by runtimes Int a = 1 + 2; 0xaaa Args: … 2 RetAddr: … 1 0xabb H E L L O \n Locals:… … … Value Stack Frame Stack Stacks 0x00 0x05 0xbbb … Local.get Local.get I32.add … Store non-num values (e.g., array and string) Next instruction address Linear Memory Program Counter 9
10. Wasm C/R Among Heterogeneous Platforms (1/2) Challenge: Depends on runtime and optimization Linear Memory Program Counter Linear Memory Program Counter ≠ locals:… locals:… module:… Frame Stack 0xFFFFFFFF module:… Internal:… 0xFFFFFF FFFFFFFFFF 32bit Value Stack Frame Stack 64bit Value Stack VM of Runtime A VM of Runtime B Diverse spec implementations between runtimes 10
11. Wasm C/R Among Heterogeneous Platforms (2/2) Challenge: Depends on runtime and optimization Program Counter locals:… module:… OS Stack 0xFFFFFFFF Program Counter Execution states in snapshot existing outside the VM itself 11
12. Our Approach: Self-Hosted Wasm Runtime-based C/R • Chiwawa: Minimal Wasm-ised Application runtime with C/R • • • Stacks Execute app via self-hosted runtime Linear Memory No need to modify host runtimes Eliminates C/R mechanisms Program counter C/R mechanism Dumped Snapshot for each optimization strategy • Chiwawa The snapshot of an application uses only Chiwawa’s execution state Program counter Stacks Any Host Runtimes Linear Memory 12
13. Technical Issues for Self-Hosted Runtime Overhead （1/3） 1. Increase bytecode instructions executed in host runtime • An instruction in an application is transformed into multiple instructions • The instruction handler in a self-hosted runtime compiles into multiple bytecode instructions local.get … fn handle_local_get(ctx: Context){ … let val = ctx.frame.locals[i].clone(); ctx.value_stack.push(val); … } 13
14. Technical Issues for Self-Hosted Runtime Overhead （2/3） 2. Duplicated computationally expensive processing Sandbox by host runtime 2-A. Sandbox mechanism • Protect platform from bugs or malicious programs • Major reason of performance degradation Sandbox by self-hosted runtime in Wasm* App 0x00 Duplicated boundary check Duplicated validation *Matthew Kolosick, Shravan Narayan, Evan Johnson, Conrad Watt, Michael LeMay, Deepak Garg, Ranjit Jhala, and Deian Stefan. 2022. Isolation without taxation: near-zero-cost transitions for WebAssembly and SFI. Proc. ACM Program. Lang. 6, POPL, Article 27 (January 2022), 30 pages. https://doi.org/10.1145/3498688 Self-hosted runtime 0xff 0x00 0xff 14
15. Technical Issues for Self-Hosted Runtime Overhead （3/3） 2. Duplicated computationally expensive processing 2-A. Sandbox mechanism • Protect platform from bugs or malicious programs • Major reason of performance degradation VM by host runtime VM by self-hosted runtime in Wasm WASI API 2-B. WebAssembly System Interface (WASI) • To access host resources • • (e.g., files and network sockets) Many stack and memory operations Self-hosted runtime WASI API by a single WASI function call 15
16. Approach for Technical Issue 1: Merging Instructions Reduce stack operations executed frequently • Almost all instructions use the value stack • Push/Pop operands (e.g., variables) and instruction results • Parser identifies dependencies between instructions • Convert stack instructions to Immediate Value … i32.const 10 local.get 0 I32.add local.set 1 … … i32.add{10, local@0} => local@1 … 16
17. Approach for Technical Issue 2-A : Reducing Sandbox Use only the sandbox created by a host runtime • • Chiwawa does not create a sandbox Sandbox by host runtime for each VM Sandbox by self-hosted runtime Maintain isolation by host runtime sandboxing • Instruction handlers of Chiwawa runs within a host sandbox App Use only hosts’s Sandbox Self-hosted runtime 17
18. Approach for Technical Issue 2-B: Pass-through WASI Minimal WASI implementation designed to be Wasm-ized • Chiwawa can invoke host’s WASI functions VM by self-hosted runtime • • VM by host runtime Compiled into Wasm itself Chiwawa’s WASI function Thin-WASI API only invokes the host's WASI function • Use arguments in the value stack or linear memory directly • Reduce data copy Self-hosted runtime WASI API 18
19. Evaluation from Three Aspects 1. Performance benefits of the optimized self-hosted runtime 2. Performance overhead introduced by Chiwawa 3. Benefits of self-hosted runtime-based C/R OS Ubuntu 22.04.4 LTS(Linux 5.15.0) CPU Intel Xeon Silver 4208 8Core 2.10GHz Memory 32 GB Storage SSD 480GBx2 (RAID1) Benchmark environment 19
20. 1. Performance Benefits of the Optimized Self-hosted Runtime (1/2) • Objective: Can Chiwawa outperform self-hostable runtimes? • Compared with Wizard* • • Fast interpreter and self-hostable Wasm runtime • Also used as a host runtime Used computationally intensive benchmarks • Leibniz formula for PI( n =1,000,000) • N-body (particle = 10,000) *Ben L. Titzer. 2022. A fast in-place interpreter for WebAssembly. Proc. ACM Program. Lang. 6, OOPSLA2, Article 148 (October 2022 ), 27 pages. https://doi.org/10.1145/3563311 20
21. 1. Performance Benefits of the Optimized Self-hosted Runtime (2/2) Reduce 71 - 85% 1.3 – 1.4x faster by minimal runtime implementation by optimization techniques Benchmarks Wizard on Wizard Chiwawa on Wizard Benchmarks Wizard on Wizard Chiwawa on Wizard pi-Leibniz 26.791 19.620 pi-Leibniz 106.98 15.03 N-body 14.554 9.932 N-body 53.46 15.15 Execution time (Sec.) Memory usage – Peak RSS(Resident Set Size) (MB) 21
22. 2. Performance Overhead Introduced by Chiwawa (1/2) • Objective: How much performance overhead is introduced by duplicated runtime execution? • Compared chiwawa on wastime with wasmtime directly • Used SQLite benchmark* • SQLite is a lightweight DB widely used in edge • Invokes many WASI functions *SQLite3 Benchmark, https://github.com/ukontainer/sqlite-bench 22
23. 2. Performance Overhead Introduced by Chiwawa (2/2) • Execution time increased in all benchmarks • 13.64x 〜426.95x • Runtime duplication increases instructions for complex operation Overhead (normalized to wasmtime) 23
24. 2. Performance Overhead Introduced by Chiwawa (2/2) • Execution time increased in all benchmarks • 13.64x 〜426.95x • Runtime duplication still exist in complex processing Complex benchmarks cause worse overhead Overhead (normalized to wasmtime) 24
25. 3. Benefits of Self-Hosted Runtime-Based C/R (1/4) • Objective: Can Chiwawa perform C/R more efficiently than existing methods? • Snapshot size with Checkpoint/Restore In Userspace (CRIU) • A C/R mechanism for Linux processes • • Achieves runtime neutrality by C/R on a Wasm runtime process Host runtime: Wasmtime, interpreter/AOT WAMR, and interpreter WasmEdge 25
26. 3. Benefits of Self-Hosted Runtime-Based C/R (2/4) • Objective: Can Chiwawa perform C/R more efficiently than existing methods? • Snapshot size with Checkpoint/Restore In Userspace (CRIU) • Significantly A C/R mechanism forstate Linux processes reduce the size • size Achieves Keep the across runtime differentneutrality runtimes by C/R on a Wasm runtime process • Host runtime: Wasmtime, interpreter/AOT WAMR, and interpreter WasmEdge Hosts Chiwawa CRIU Wasmtime 1076 9232 WAMR(Interp.) 1076 654484 WAMR(AOT) 1076 14360 WasmEdge 1076 89784 Checkpointed Wasm-bytecode state size (KB) 26
27. 3. Benefits of Self-Hosted Runtime-Based C/R (3/4) • Objective: How do C/R mechanisms impose app performance overhead? • Compared application performance differences with and without the C/R mechanism in Chiwawa and Runtime-based C/R • Comparison target: Wizard • Can monitor execution states required for C/R* • Emulated runtime-based C/R by tracing these while running the application • Used SQLite benchmark • Host runtime: Wasmtime *Ben L. Titzer, Elizabeth Gilbert, Bradley Wei Jie Teo, Yash Anand, Kazuyuki Takayama, and Heather Miller. 2024. Flexible Non -intrusive Dynamic Instrumentation for WebAssembly. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 (ASPLOS '24), Vol. 3. Association for Computing Machinery, New York, NY , USA, 398–415. https://doi.org/10.1145/3620666.3651338 27
28. 3. Benefits of Self-Hosted Runtime-Based C/R (4/4) Overhead of emulated runtime-based C/R Overhead of Chiwawa’s C/R (normalized to wizard without C/R) (normalized to chiwawa without C/R) 28
29. 3. Benefits of Self-Hosted Runtime-Based C/R (4/4) Runtime-based C/R may introduce significant overhead Monitoring execution state is a costly operation Overhead of emulated runtime-based C/R Overhead of Chiwawa’s C/R (normalized to wizard without C/R) (normalized to chiwawa without C/R) 29
30. 3. Benefits of Self-Hosted Runtime-Based C/R (4/4) Chiwawa’s C/R does not significantly degrade performance Self-hosted runtime-based C/R is efficient Overhead of emulated runtime-based C/R Overhead of Chiwawa’s C/R (normalized to wizard without C/R) (normalized to chiwawa without C/R) 30
31. Conclusion • Live migration by C/R between edge and cloud is a key factor for application offloading and handoff • Chiwawa: a self-hosted runtime-based C/R with optimization tailored for self-hosted • • Eliminate runtime and optimization differences in C/R • Enable efficient C/R with a minimal snapshot and overhead • Current performance optimization is insufficient and must be improved Future work: • Further performance optimization for the Self-hosted runtime • C/R containing WASI execution state existing outside the Wasm runtime 31
32. Existing Approaches Suffer from complexity and constraints Runtime-based C/R[2] Bytecode Bytecode C/R for A C/R for B Runtime A Runtime B Complex state translation per runtime 32
33. Existing Approaches Suffer from complexity and constraints Runtime-based C/R[2] Bytecode Bytecode C/R for A C/R for B Runtime A Runtime B Bytecode translation[3] Invoke C/R Functions … call $cr Insert Codes local.get $p1 … call $cr local.get $p1 local.get $p2 local.get $p2 call $cr i32.add i32.add call $hoge call $cr … … Complex state translation Vast numbers of function calls per runtime or limitations on C/R-enabled points 33