What containerd 2.0 means for you (KubeCon + CloudNativeCon North America 2024

containerd 2.0 is the first major new version of containerd since 1.0.0 was released in 2017. This new version of containerd introduces new features, new extension points, and new backends for image operations and CRI with the goal of increased flexibility and better efficiency for certain types of workloads. containerd 2.0 also removes some previously-deprecated features in favor of modern replacements. This talk will discuss how to prepare for containerd 2.0 in your production environments, including strategies for incorporating containerd 2.0's new functionality and detecting/remediating any impact of removed features prior to upgrading.

Transcript

2. What containerd 2.0 means for you Samuel Karp, Google
3. containerd 2.0 is now available!
4. • What’s new • What’s changing • Deprecations and removals • Preparing to upgrade • How you can remediate any issues • Where to find help • What’s next • Q&A Agenda
5. So what’s new?
6. • Transfer service NOW STABLE • Sandbox service (and sandboxed CRI) NOW STABLE • Faster image extraction with igzip NEW • Improved OTEL configuration NEW • NRI enabled by default NEW • Image verifier plugins NEW • Plugin introspection NEW • CDI enabled by default NEW • CRI support for user namespaces NEW New features, newly stable features, defaults
7. • Akin to a mutating webhook, but for container configuration ◦ Middleware between CRI and OCI • Use cases ◦ Injection (devices, network devices, OCI hooks) ◦ Resource modification/management (ulimits, topology/NUMA, advanced QoS, SGX memory) ◦ Policy enforcement • Plugins can run in containers or as system services • Enabled by default • Community plugins ◦ https://github.com/containerd/nri/tree/main/plugins ◦ https://github.com/containers/nri-plugins Highlight: Node Resource Interface (NRI)
8. • Exec-based plugins containerd invokes during image pull • Policy enforcement use-cases ◦ Container image signature verification ◦ Trust for particular signers ◦ Allow only specific registries/repositories • Integrated with the Transfer service (not supported for legacy pulls) Highlight: Image verifier plugins
9. Changes (non-breaking)
10. • Go client library as a separate package ◦ Now in github.com/containerd/containerd/v2/client • gRPC API is versioned separately from containerd ◦ Go package github.com/containerd/containerd/api • CRI registry properties are deprecated (but not removed) ◦ mirrors, auths, and configs of [plugins.\"io.containerd.grpc.v1.cri\".registry] ◦ Targeted for removal in 2.1 ◦ auths replaced by credential manager plugins • Go-plugin (*.so) libraries are deprecated ◦ Targeted for removal in 2.1 • Envelope type is changing ◦ types.Envelope replaces service.events.Envelope and ttrpc.events.Envelope ◦ Targeted for removal in a future release Changes
11. Deprecations ⇒ Removals (breaking changes)
12. Removals can be scary But don’t worry We’ll talk about how to prepare next
13. • Docker Schema 1 image support DISABLED BY DEFAULT • CRI v1alpha2 API • Default LimitNOFILE in systemd unit • io_uring_* syscalls in default seccomp profile • io.containerd.runtime.v1.linux and io.containerd.runc.v1 • containerd.io/restart.logpath label • AUFS snapshotter • CRI+CNI release bundles Removals
14. Preparing to upgrade (finding out if you need to take action)
15. • Deprecation warnings exposed by containerd ◦ Usage-based; warnings show up if usage is detected ◦ If you see the warning, you should expect to take action • Use ctr deprecations list to see warnings (optional: --format json) • The lastOccurrence field records the last time the deprecated feature was used.* • Warnings are cleared when containerd is restarted • Use at least containerd 1.7.12 or 1.6.27 * Config-related warnings will show the time containerd started. The CRI v1alpha2 warning will show the first occurrence for containerd versions before 1.7.21 and 1.6.36. Finding out about impact
16. • ctr deprecations list • Supported in v1.7.12+, v1.6.27+ • For each warning, you can see ◦ What is being removed ◦ A suggestion for migration ◦ When it was last used Deprecation warnings
17. 1. Upgrade to the latest 1.7.x or 1.6.x release of containerd a. (prefer at least 1.7.21 or 1.6.36) 2. Find your impact through deprecation warnings 3. Remediate the warnings a. Config format v1 will be auto-migrated; you can convert fully later 4. Test to make sure no warnings reoccur (restart containerd or create new nodes) 5. Try upgrading to containerd 2.0 (test clusters are a good practice) 6. Upgrade when you are ready • containerd 1.6 will be supported until the next LTS • containerd 1.7 will transition to Extended support in 6 months Upgrade strategy
18. • Providers can integrate warnings into their systems for you • Case study: Google Kubernetes Engine ◦ containerd 2.0 is currently expected in GKE 1.33 ◦ Deprecation warnings exposed as Node Conditions ◦ Recommendations surfaced in the Google Cloud console (coming soon) ◦ Automatic upgrades to containerd 2.0 will be blocked if deprecated feature use is detected Providers can help
19. Remediation (selected highlights)
20. • Media type: application/vnd.docker.distribution.manifest.v1+prettyjws • Replaced by Docker schema 2 (2014) and OCI Images (2017) • Most modern images are not schema 1 • containerd labels schema 1 images for you ctr image list 'labels."io.containerd.image/converted-docker-schema1"' • containerd can convert for you; pull + push image to fix • Schema 1 support can be re-enabled, but will be removed in 2.1 Highlight: Docker Schema 1 images
21. • CRI v1 replaced v1alpha2 in Kubernetes 1.23 • Kubelet should use the correct version • Other workloads may still depend on v1alpha2 • Use containerd 1.7.21 and 1.6.36 for accurate warnings Highlight: CRI v1alpha2
22. • Not a deprecation • Sandboxed CRI replaces legacy CRI • Internal refactor of CRI support on top of the new Sandbox and Sandboxer abstractions • Try it out in 1.7 with ENABLE_CRI_SANDBOXES=true in containerd.service Highlight: Sandboxed CRI
23. • Most deprecations have direct replacements • https://containerd.io/releases/#deprecated-features • Deprecation warnings will also provide suggestions • Most applications should degrade gracefully for io_uring changes The rest of the changes
24. • Guide to containerd 2.0: https://github.com/containerd/containerd/blob/main/docs/container d-2.0.md • Discussions: https://github.com/containerd/containerd/discussions • Slack: #containerd on cncf.slack.io • Community meetings: cncf.io/calendar (2nd and 4th Thursdays) Finding help
25. What’s next (2.1 and beyond)
26. • 2.1 milestone: https://github.com/containerd/containerd/milestone/48 • OCI Image Volume Source (KEP-4639) • Continuing improvements to Sandboxes/Sandboxers • Image pull improvements (multi-part layer fetch) • Integration for CRI with the Transfer service • Credential manager plugins for Transfer service What’s next
27. containerd maintainer session “What’s going on in the containerd neighborhood” 2:55pm-3:30pm Hyatt Regency | Level 4 | Regency Ballroom A
28. Q&A Join us on Slack: #containerd and #containerd-dev at slack.cncf.io Community meetings on cncf.io/calendar github.com/containerd/containerd
29. Feedback provides valuable information to speakers (and I’d really appreciate yours)! Feedback that is very helpful: • Topics you were excited to learn about • Suggestions for improving understanding and clarity Please avoid comments unrelated to the talk subject or content (refer to the CNCF Code of Conduct) A brief note before we finish –
30. Follow the QR code to leave session feedback!