Possibility of OCI Container Runtime with Rust

It goes without saying that the Rust language has received a lot of attention in recent years, including the adoption of Rust as a second language in the Linux kernel. Toru has been exploring the possibility of developing a container runtime using Rust in the vicinity of OCI Runtime and developing youki, the OCI container runtime in Rust that He has worked on for over a year and a half. He thought it would be a good fit for the language since OCI Runtime requires development at a relatively kernel-like layer. Youki is now listed as one of the reference implementations of the OCI Runtime specification and is at a level where it can be used from docker and podman. The next step is to consider using it with Kubernetes. In this session, He will discuss the possibility of using Rust in a container runtime. He will share his experiences with youki. From these experiences, He feels that Rust is a language that should contribute more to the container runtime field. At the beginning of the presentation, I will explain container technology such as cgroups and Linux namespaces for those who have not had much exposure to OCI Runtime.

At KubeDay Japan
https://events.linuxfoundation.org/kubeday-japan/
https://sched.co/1C8lS

Transcript

1. Possibility of OCI Container Runtime with Rust Toru Komatsu (@utam0k) Senior Enginner at Gitpod 1
2. Toru Komatsu, Senior Engineer at Gitpod utam0k utam0k A member of containers organization I develop youki as a hobby 2
4. What I'd like to tell 01 4
5. Today you will learn Based on our experience of developing youki, the OCI Runtime in Rust, I'll introduce the compatibility and attractiveness of Rust and OCI Runtime. And about the current status and future of youki. 5
6. Agenda 01 What I'd like to tell 02 What is OCI Container Runtime? 03 Why Rust? 04 Where we are now 05 Problems and Future 6
7. What is OCI Container Runtime? 02 7
8. Kubelet(K8s) Container creation flow from kubelet Linux etc… High-Level Runtime CRI Low-Level Runtime runc runsc kata OCI Container Runtime 8
9. How do we create containers? pivot_root(2) Change a destination that the root directory of a process points to namespace(7) Ability to isolate resources that a process can manipulate cgroup Allows configuration of resources available to processes(containers) 9
10. Youki OCI Container Runtime in Rust Developed since January 2021 Vendor Neutral Under the Containers organization, which manages podman, etc. https://github.com/containers/youki 4.0K ⭐ on GitHub Provide a Rust library for OCI from a part of youki youki means a container in Japanese 10
11. Why Rust? 03 11
12. Benefits of using Rust Close to kernel WASM Lighter-weight 12
13. Benefits of using Rust WASM Lighter-weight Close to kernel Avoiding namespace restrictions 13
14. setns(2) - reassociate thread with a namespace A multithreaded process may not change user namespace with setns(). 14
15. netns(2) with Go HighLevelCR HighLevelCR runc create Only Go runc create Only Go runc init C&Go runc init C&Go Create a container Prepare to create a container such as cgroup /proc/self/exec init with args as ENVs Actually create a container 15
16. netns(2) with Go HighLevelCR HighLevelCR runc create Only Go runc create Only Go runc init C&Go runc init C&Go Create a container Prepare to create a container such as cgroup /proc/self/exec init with args as ENVs Actually create a container 16
17. netns(2) with Go HighLevelCR HighLevelCR runc create Only Go runc create Only Go runc init C&Go runc init C&Go Create a container Prepare to create a container such as cgroup /proc/self/exec init with args as ENVs Actually create a container 17
18. netns(2) with Go HighLevelCR HighLevelCR runc create Only Go runc create Only Go runc init C&Go runc init C&Go Create a container Prepare to create a container such as cgroup /proc/self/exec init with args as ENVs Actually create a container 18
19. • • • • • There is no limit around the namespace(7) Rust can be implemented alone with a minimum number of clones Fewer potential security holes because no extra operations are required CVE-2019-5736 does not happen with Rust implementation Rust has been adopted as a second language for Linux Ⓒ The Rust foundation 19
20. • • buddy-buddy How to use → https://containers.github.io/youki/user /webassembly.html Ⓒ The Rust foundation 20
21. Ⓒ The Rust foundation 21
22. Benefits of using Rust Close to kernel WASM Lighter-weight 22
23. Runtime Language Time (mean ± σ) Range (min … max) crun C 153.5 ms ± 21.6 ms 80.9 ms … 196.6 ms youki Rust 198.4 ms ± 52.1 ms 97.2 ms … 296.1 ms runc Go 352.3 ms ± 53.3 ms 248.3 ms … 772.2 ms Time from container startup to deletion 23
24. What kind of future can you imagine with lightweight container runtime? IoT? running it in a car? lighter-weight 24
25. Benefits of using Rust Close to kernel Avoiding namespace restrictions WASM New Possibilities Lighter-weight Less memory used, better performance 25
26. Why Go? Why Rust? To begin with, runc was initially part of Docker and spun out from it. Much used in this community and has a lot of history. Go is mature in this area. Therefore, libraries are abundant But there is a problem that cannot be solved around the namespace. In other words, it cannot be implemented by Go alone, strictly. Why Go? Simple, safe, and incidentally light However, there is not much history and libraries. Why Rust? 26
27. Where we are now 04 27
28. Fighting with the real • ✅ Test for OCI Runtime prepared by OCI ✅ Containerd integration test Achieved primarily through @YJDoc2 28
29. For Kubernetes ✅ kubeadm supported youki 29
30. Advanced Features ✅ WASM ✅ cgroup v2 ✅ Rootless 30
31. Problems and Future 05 31
32. Problems Old kernels Experience 32
33. Problems Old kernels Experience 33
34. We are creating the future Ideally, we have to support old Linux kernels, but it is challenging, and other container runtimes have already been supported. We would like to imagine beautiful futures with the latest kernel technology instead of dropping them off. 34
35. Problems Old kernels Experience 35
36. Standing on the shoulders of giants Let's use the history of OCI Container Runtime pioneers! OCI Runtime is just a binary. Let's cheat on the name and use the giant test. 36
37. Roadmap/ upcoming tasks/ events 2023・2024 Pass the integration test of runc and others containers/youki#1305 OpenTelemetry tracing support to extend observability containers/youki#1348 Contribute to WASM world containers/youki#1320 Consider ideas using the latest technologies such as eBPF and io_uring 37
39. runwasi • • • • • • • • Low-level container runtime designed for WASM in Rust Still not meeting OCI Runtime Spec Used by Docker WAM Azure Kubernetes Service for WASM https://learn.microsoft.com/en-us/azure/aks/use-wasi- node-pools repo: containers/runwasi First committed by Deis Labs, a Microsoft group company 39
40. youki 　 　runwasi 40
41. Thanks to all the heroes who contributed to youki 41
42. Thanks! Any question? Toru Komatsu utam0k utam0k github.com/containers/youki 42