Revamping VRoid Hub’s model processing pipeline with Rust

videos:
- https://vimeo.com/1013050955
- https://vimeo.com/1013052237

Transcript

1. Revamping VRoid Hub’s model processing pipeline with Rust yue
2. Frontend & 3D developer VRoid team since 2021 I enjoy type systems! yue
3. Overview
4. Introduction
6. This talk focuses on
7. This talk focuses on
8. This talk focuses on
9. Over 2.8 million 3D avatar models have been posted to VRoid Hub
10. System architecture
11. VRoid Hub’s Architecture (Before)
12. VRoid Hub’s Architecture (Before)
13. VRoid Hub’s Architecture (Before)
14. VRoid Hub’s Architecture (Before)
15. VRoid Hub’s Architecture (Before)
16. VRoid Hub’s Architecture (Before)
17. server-side components
18. server-side components
19. server-side components
20. server-side components
21. VRoid Hub’s Architecture (Before)
22. How to improve?
23. Our journey
24. How about Rust…?
25. • Performances ✅ • Control and correctness ✅ • Wide range of build targets ✅ • Good variety of libraries with quality ✅ • Team members' skills ✅ Why Rust?
26. The first step
27. • small code base ✅ • easy to POC / prototype ✅ • easy to compare ✅ ◦ we did the comparison with Node.js / golang.. • easy to try and error (nothing is stored)✅ Why start from the download server?
28. Transport
31. VRMFLUX • Average response time: 4x faster. • CPU usage: about 1/4. LGTM!
32. Step.1: Convert vrmflux to rust ✅
33. The second step
34. Dev meetup talk 3 years ago...
35. VRoid Hubにおける 3Dモデルの最適化 pixiv Inc. Isamu Mogi
36. ポータブルな Unityアプリ実行環境の構築 LLVMpipe Mesa Xvfb Unity App GNU C Library Linux Kernel GPU エミュレー ション OpenGL GUI printf(); 一番偉い ポータブルにするには 自前で準備する必要あり セットで存在する LinuxサーバーでのUnityアプリケーション実行環境の模式図 → too complex 😣
37. ポータブルな Unityアプリ実行環境の構築 ビルドに必要なライブラリ https://gitlab.freedesktop.org/xorg/util/modular/-/blob/8cd77d79b12cb802a3eda0fdd6f7ce11662d02 e2/build.sh#L1029 util-macros,font-util,xorgproto,xcb-proto,xtrans,libXau,libXdmcp,libpthread-stubs,libxcb,xcb-util,xcb- util-image,xcb-util-keysyms,xcb-util-renderutil,xcb-util-wm,libX11,libXext,libdmx,libfontenc,libFS,libI CE,libSM,libXt,libXmu,libXpm,libXau,libXaw3d,libXfixes,libXcomposite,libXrender,libXdamage,libXc ursor,libXfont2,libXft,libXi,libXinerama,libxkbfile,libXrandr,libXres,libXScrnSaver,libxshmfence,libXtst ,libXv,libXvMC,libXxf86dga,libXxf86vm,libpciaccess,pixman,libdrm,mesa,xbitmaps,xkbcomp,xorg-s erver,xkeyboard-config すごく多い・・・ → cargo build ✅
38. • Prepare for advanced processing ◦ perform texture atlas in pure rust context ◦ build an importer & exporter for glTF/VRM from scratch The challenge
39. with Rust
40. The glTF™ format • GL Transmission Format • png for 3D models • simple: json + binary • optimized for transmission/runtime
41. glTF uses indexes to define relationships
43. glTF is a property graph
44. If we virtualize the graph..
45. a method for improving rendering cost Texture atlas
46. Before
47. After
48. Find an edge with the relationship Modify the graph in O(1) time
49. Step2: Convert vrmio/vrm-ruby to rust ✅
51. VRMIO • A stable glTF/VRMimporter/exporter • Easier to build, test, and maintain
52. Shared Engine
53. • compiled to wasm/wasi targets ◦ runs in browser ◦ runs in docker WasmEdge Runtime • compiled as bindings ◦ node w/ napi-rs ◦ ruby w/ magnus Some future possibilities
54. DEMO: VRMFLOW
55. Node-based 3D model editor using the same engine
56. Libraries & Testing
57. hyper and AWS SDK for Rust reached v1
58. • gltf_rs / glTF handling ◦ used in Bevy engine (game engine) • petgraph / graph processing ◦ the same tool behind swc (js build tool) • euclid & guillotiere / math & atlas implementation ◦ comes from Servo community (browser engine) Crates we used
59. Powered by game and browser engines!
60. • salsa / on-demand, incrementalized computation. ◦ powers rust-analyzer (Rust LSP) • KTX-Software / meshoptimizer ◦ unlocks the possibility to link with C/C++ code More dependencies available
61. • distroless / docker base image • cargo deny / license and security scan Extra tooling we found useful
62. • insta / snapshot testing • playwright / E2E testing with WebGL • Sentry (has an official Rust crate!) / error tracking • Datadog / performance monitoring How did we test
63. We even built a dedicated real-time testing tool
64. OSS contributions
65. We contribute back to the ecosystem Three.js / @types/three / gltf-Transform / ktx-parse / basis-universal-rs / vrm-utils-rs / …
66. We maintain OSS projects • three-vrm • vrm-utils-rs new! ◦ Data structures for the VRM Format. ◦ published on GitHub / crates.io ◦ https://github.com/pixiv/vrm-utils-rs
67. We are a part of the VRM Consortium
68. Summary
69. To wrap it up • At VRoid, we will continue to focus on improving existing systems and user experiences. • There are more plans and features coming this year to enhance our services. Please stay tuned!
70. Thanks!