Beyond the buzzword: BPF's unexpected role in Kubernetes

Presented jointly with Alban Crequy, Kinvolk co-founder and director of Kinvolk Labs, at KubeCon NA 2020.

Increasingly, cloud native tools are leveraging the Linux kernel’s Berkeley Packet Filter (BPF) capabilities for a range of applications, such as networking, security, observability, and troubleshooting. In recent Linux kernel releases, BPF has been significantly enhanced, resulting in this proliferation of BPF-powered cloud native projects. In this talk, we will go beyond the BPF buzzword, review the latest relevant developments in the Linux kernel and the rich ecosystem of cloud native tooling built on it. With live demos we will show how easy it is to deploy these BPF-based tools into your Kubernetes cluster, gaining greater visibility and control at the pod level of granularity.

Transcript

1. Beyond the Buzzword: BPF’s unexpected role in Kubernetes November 19, 2020 Andrew Randall Alban Crequy
2. What is (e)BPF? custom programs that run in the Linux kernel hooks and data structures (maps) restricted virtual machine sandbox + code verifier (extended) Berkeley Packet Filter
3. Why do you care? fast, customizable networking debugging / performance analysis application monitoring & security
4. Evolution of (e)BPF 2.1.75 first BPF support Dec 1997 3.15 new JIT compiler → eBPF Jun 2014 IO Visor project established Aug 2015 4.8 eXpress Data Path (XDP) Oct 2016 4.11 BPF datastructures for improved packet filtering May 2017 May 2018 Katran announced by Facebook Nov 2017 4.14 fast intra- host networks (sockmap) 4.18 bpf filter by cgroups (containers) Aug 2018 Aug 2020 5.8 BPF ring buffers
5. An eBPF OSS Landscape Low-level tools Security & Networking Visibility bcc bpftrace cilium falco katran llvm API Libraries gobpf ebpf libbpf libbpf-rs red-bpf calico polycube skydive hubble weave scope kubectl- trace kubectl- gadget kernel tools e.g. bpftool tcptracer-bpf Other ply pyebpf
6. Hubble
7. Hubble
8. Weave Scope
9. IOvisor BPF Compiler Collection (bcc)
10. bpftrace
12. Enter: Inspektor Gadget a “swiss army knife” collection of various bpf tools (gadgets) some from bcc + some new ones developed by kinvolk
13. What do we need for Kubernetes? granularity: “pod, not pid” aggregation by label selectors kubectl-like experience
14. K8s integration My laptop $ kubectl gadget... kubectl-gadget Kubernetes Control Plane (API Server, scheduler, ...) exec client plugin worker node “gadget” pod exec traceloop & bcc kernel Install BPF program Deploy gadget pods Kubernetes cluster Create DaemonSet kubectl-exec
15. Gadgets available today profile network policy advisor traceloop tcptop tcptracer opensnoop execsnoop bindsnoop capabilities kubectl-gadget
16. Demo kubectl demo