cgroup v2 support in Kubeadm

1. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. cgroup v2 support in Kubeadm 2024年 10月 10日 多田 健太 トヨタ自動車株式会社 情報通信企画部 InfoTech-IS 兼オープンソースプログラムグループ 主幹 Kubernetes Meetup Tokyo #67 1
2. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. X : @kenta_tada Toyota Motor Corporation • Project Manager ◦ Develop both server-side and automotive systems ◦ Accelerate eBPF adoption across the company • Open Source Program Office in Toyota Recent Activities • eBPF Japan Community Organizer • KubeDay Japan 2024 Co-chair • The reviewer of kubernetes/system-validators • Cloud Native Community Japan Organizer • The reviewer of the Japanese translation of the book "Learning eBPF" published by O'Reilly Japan Kenta Tada
3. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. [PR] eBPF Japan Meetup #2 開催決定!! ⚫ 12/6(金)に、さくらインターネット様の東京支社をお借りして、 eBPF Japan Meetup #2 を開催します! ⚫ LT 枠の登壇者 2 名募集中!!(X 上で多田までご連絡ください) ⚫ 聴講参加枠は現在準備中です。もう少しお待ちください。 3 8/19(月) に開催した eBPF Japan Meetup #1
4. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. Agenda ⚫ cgroup v1 maintenance mode ⚫ Pre-flight checks of Kubeadm ⚫ Challenges for pre-flight checks of cgroup v2 4
5. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. cgroup v1 maintenance mode (KEP-4569) ⚫ Since Kubernetes 1.31, if the host is running on cgroup v1, kubelet will log a warning message. ⚫ Goals ⚫ Feature Freeze ⚫ e2e Testing ⚫ Security Maintenance ⚫ Best-Effort Bug Fixes ⚫ Migration Support ⚫ Non-Goals ⚫ Removing cgroup v1 support. ✓ Deprecation and removal will be addressed in a future KEP. 5
6. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. Problem with incompatible workloads ⚫ This KEP helps users using cgroup v1 to migrate to cgroup v2. ⚫ But users need to migrate to cgroup v2 about their own workloads. ⚫ For example, users depending on the following technologies will need to ensure the support of cgroup v2: ⚫ OpenJDK / HotSpot: jdk8u372, 11.0.16, 15 and later ⚫ NodeJs 20.3.0 or later ⚫ If users run their own tools that depend on the cgroup file system or kernel APIs related to cgroup v1, they need to update those tools to support cgroup v2. 6
7. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. Pre-flight checks of Kubeadm ⚫ Kubeadm is a tool built to provide best-practice "fast paths" for creating Kubernetes clusters. ⚫ Pre-flight checks of Kubeadm ⚫ kubeadm init executes some pre-flight checks to verify preconditions and avoid cluster’s startup problems. ⚫ Kubeadm uses kubernetes/system-validators for Kubeadm pre-flight checks. ⚫ During pre-flight checks, Kubeadm also checks features available in cgroup. ⚫ But the current implementation is not enough to check features available cgroup v2. 7
8. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. Challenges for pre-flight checks of cgroup v2 ⚫ How to detect features available cgroup v2 correctly ⚫ When it comes to v2, /proc/cgroups is meaningless. ⚫ Some features cannot be determined from the kernel configuration. ⚫ Understanding kernel configuration dependencies accurately ⚫ It is difficult to determine the truly necessary kernel configurations for each cgroup version. ⚫ For example, Utilization Clamping which is supported in v2 depends on CONFIG_CGROUP_SCHED. ⚫ Being careful about backward compatibility ⚫ Most OS distributions keeps the cgroup v1 related kernel config as is. 8
9. Copyright © 2024 TOYOTA MOTOR CORPORATION All rights reserved. References ⚫ https://github.com/kubernetes/enhancements/tree/master/keps/sig- node/4569-cgroup-v1-maintenance-mode, (Accessed on October 4, 2024) ⚫ https://kubernetes.io/blog/2024/08/14/kubernetes-1-31-moving-cgroup- v1-support-maintenance-mode, (Accessed on October 4, 2024) ⚫ https://github.com/kubernetes/system-validators, (Accessed on October 4, 2024) ⚫ https://github.com/kubernetes/enhancements/issues/4569, (Accessed on October 4, 2024) 9