Upgrade to Pro — share decks privately, control downloads, hide ads and more …

gobpf - utilizing BPF from Go - Alban Crequy - ...

GoDays
January 23, 2020

gobpf - utilizing BPF from Go - Alban Crequy - Kinvolk

BPF is a Linux in-kernel virtual machine that is used for tracing kernel functions, networking, performance analysis and more. In this talk I will give an introduction into eBPF and gobpf, a library under the IO Visor Project, and show how gobpf can be used to access and utilize BPF features from Go programs. We will also look at the design of gobpf and its use of the bcc framework and Cgo.

GoDays

January 23, 2020
Tweet

More Decks by GoDays

Other Decks in Technology

Transcript

  1. The Kubernetes Linux Experts Engineering services and products for Kubernetes,

    containers, process management and Linux user-space + kernel Blog: kinvolk.io/blog Github: kinvolk Twitter: kinvolkio Email: [email protected] Kinvolk
  2. @albcr Plan - Introduction to BPF - BPF Compiler Collection

    (BCC) - gobpf - Demos - Future features
  3. @albcr Intro to Berkeley Packet Filter - BPF: bytecode executed

    in the Linux kernel - Initially for tcpdump (1992) - Extended BPF (2013)
  4. @albcr How tcpdump captures network packets Local process (apache) input

    output routing decision forward post-routing pre-routing Incoming packet tcpdump socket AF_PACKET copy socket AF_INET
  5. @albcr tcpdump filters - Not Turing complete, but - Complex

    and arbitrary filters: tcpdump src 10.0.2.4 and (dst port 3389 or 22) tcpdump portrange 21-23 tcpdump 'tcp[13] & 1!=0' tcpdump 'tcp[32:4] = 0x47455420' (HTTP “GET ”)
  6. @albcr tcpdump filters Incoming packet tcpdump socket AF_PACKET Local process

    (apache) input output routing decision forward post-routing pre-routing bpf filter port=22 socket AF_INET userspace kernel
  7. @albcr Classic BPF (1992) -> extended BPF (2013) - Different

    kinds of BPF programs - Not just for tcpdump anymore - bpf() system call - BPF maps - BPF filesystem: /sys/fs/bpf
  8. @albcr ...attached to different objects Different kinds of BPF programs...

    Networking Security Tracing sockets (tcpdump) (Landlock), KRSI kprobes cgroups seccomp tracepoints traffic control devices uprobes
  9. @albcr The BPF verifier ensures it’s safe SEC("socket/0") int main_prog(struct

    __sk_buff *skb) { bpf_map_lookup_elem(...) return 0; } input parameters Output: return value BPF helper functions (e.g. lookup maps) no loops x x access to kernel memory
  10. @albcr BPF Maps kernel userspace BPF program 1 BPF map

    Userspace program ∘ Keep context between calls ∘ Report to userspace API: syscall bpf(cmd, ...) - BPF_MAP_CREATE - BPF_MAP_LOOKUP_ELEM - ... API: helper functions - bpf_map_lookup_elem() - bpf_map_update_elem() - bpf_map_delete_elem() BPF program 2
  11. @albcr BPF Maps - 25 different types today, including: -

    Hash table - Array - Queue - Perf ring buffers - Created with: - fd = bpf() system call - Command line: “bpftool map create”, visible on bpf filesystem (/sys/fs/bpf)
  12. @albcr BCC - “BCC makes BPF programs easier to write,

    with kernel instrumentation in C (and includes a C wrapper around LLVM), and front-ends in Python and lua.” - Lots of examples and tools - https://github.com/iovisor/bcc - https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
  13. @albcr gobpf history - Some original code from PLUMgrid &

    IO Visor - In 2016: Kinvolk adds a feature with BPF in Weave Scope (in Golang and not Python) - Grew organically in features - Now 42 contributors, top 4 at Kinvolk - No releases yet. Currently recommend to vendor the library
  14. @albcr gobpf: two libraries Library to create, load and use

    eBPF programs from Go • use Cgo + the BPF Compiler Collection (bcc) import "github.com/iovisor/gobpf/bcc" or • load and use pre-built elf object files import "github.com/iovisor/gobpf/elf"
  15. @albcr DEMO gobpf (bcc) • bashreadline.py but in Golang sudo

    -E go run examples/bcc/bash_readline/bash_readline.go • execsnoop but in Golang go build ./examples/bcc/execsnoop sudo ./execsnoop
  16. @albcr DEMO gobpf (elf): tcptracer-bpf • Add kprobes on kernel

    functions: ◦ TCP connect ◦ TCP accept ◦ TCP close sudo ./tests/tracer https://github.com/weaveworks/tcptracer-bpf
  17. @albcr DEMO gobpf (elf): traceloop • Tracing system calls in

    cgroups • Flight recorder with overwritable ring buffers sudo ./traceloop cgroups --dump-on-exit \ /sys/fs/cgroup/system.slice/sshd.service https://github.com/kinvolk/traceloop https://github.com/kinvolk/inspektor-gadget #inspektor-gadget on Kubernetes Slack
  18. @albcr Module Different structs / objects Kprobe Map Map Uprobe

    CgroupProgram SocketFilter SchedProgram TracepointProgram XDPProgram PerfMap Map • One module per ELF file • ELF files contains ELF sections ◦ For each map ◦ For each BPF program
  19. @albcr type Module struct • Constructor func NewModule(fileName string) *Module

    func NewModuleFromReader(fileReader io.ReaderAt) *Module • Loading func (b *Module) Load(parameters map[string]SectionParams) error - Ring buffer size - Pinning config - Override max_entries Read an ELF file
  20. @albcr type Module struct • Attaching func (b *Module) EnableKprobe(secName

    string, maxactive int) error func (b *Module) EnableKprobes(maxactive int) error func (b *Module) EnableTracepoint(secName string) error ... • Getters func (b *Module) Map(name string) *Map func (b *Module) IterMaps() <-chan *Map func (b *Module) Kprobe(name string) *Kprobe func (b *Module) IterKprobes() <-chan *Kprobe ...
  21. @albcr type Map struct • Generic read & write (hash

    table, array...) func (b *Module) LookupElement(mp *Map, key, value unsafe.Pointer) error func (b *Module) LookupNextElement(mp *Map, key, nextKey, value unsafe.Pointer) (bool, error) func (b *Module) UpdateElement(mp *Map, key, value unsafe.Pointer, flags uint64) error func (b *Module) DeleteElement(mp *Map, key unsafe.Pointer) error • Low-level functions func (m *Map) Fd() int
  22. @albcr type PerfMap struct • Helper to read the ring

    buffer using Go channels • Constructor func InitPerfMap(b *Module, mapName string, receiverChan chan []byte, lostChan chan uint64) (*PerfMap, error) • Ring buffer features func (pm *PerfMap) SetTimestampFunc(timestamp func(*[]byte) uint64) func (pm *PerfMap) PollStart() func (pm *PerfMap) PollStop()
  23. @albcr Module Flexible model Kprobe Map Map Kprobe Module Kprobe

    Map Map Kprobe • Different programs can reference the same map • Including between different modules via pinning
  24. @albcr Module “main.o” Real example with traceloop tail_call_enter tracepoint__sys_enter tracepoint__sys_exit

    tail_call_exit cgroup_map Module “tailcall.o” (1) events tracepoint__sys_enter tracepoint__sys_exit probe_at_sys_exit Module “tailcall.o” (2) events tracepoint__sys_enter tracepoint__sys_exit probe_at_sys_exit ...
  25. @albcr Examples of missing features - Better debug information from

    the verifier - DWARF debug information in ELF files - Access to kernel structs between different versions - BTF (BPF Type Format) - Support for more BPF maps - Example: maps of maps - Support for more BPF programs - Example: Linux Infrared Remote Control (LIRC)
  26. Alban Crequy Github: alban Twitter: albcr Email: [email protected] Kinvolk Blog:

    kinvolk.io/blog Github: kinvolk Twitter: kinvolkio Email: [email protected] Slides: https://tinyurl.com/godays-gobpf Thank you!