Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cilium & cgroup eBPF

Rueian
October 26, 2020

Cilium & cgroup eBPF

Tracing the linux kernel commit history to understand what is cgroup eBPF and how Cilium use it to perform NAT on system calls to replace kube-proxy's iptable rules.

Rueian

October 26, 2020
Tweet

More Decks by Rueian

Other Decks in Technology

Transcript

  1. Ruian Huang @ Dcard / Cloud Native TW Meetup #33.

    2020/10/26 Cilium & cgroup eBPF cgroup eBPF applications
  2. Hi, I am Ruian • Graduated from NCTU CSCC •

    Dcard Backend Engineer • https://medium.com/@ruian • http://github.com/rueian • https://speakerdeck.com/rueian Previous Sharing
  3. • Why Cilium? Why eBPF? • What is cgroup eBPF?

    • commit history • connect syscall example • How does Cilium use cgroup eBPF? • How does Cilium agent prepare the eBPF map? • How does Cilium eBPF program utilize the map? Outline
  4. Why eBPF? • High Performance • Able to skip large

    amount of executions in kernel. • Full Control • Able to change kernel/application behavior on the fly. Other eBPF Applications • [XDP] https://blog.cloudflare.com/how-to-drop-10-million-packets/ • [XDP] https://blog.cloudflare.com/unimog-cloudflares-edge-load-balancer/ • [XDP] https://engineering.fb.com/open-source/open-sourcing-katran-a- scalable-network-load-balancer • https://github.com/zoidbergwill/awesome-ebpf
  5. Source: KubeCon2020 - Hubble eBPF Based Observability for Kubernetes By

    Sebastian Wicki, Isovalent Where is XDP eBPF?
  6. Where is XDP eBPF? Source: KubeCon2020 - Hubble eBPF Based

    Observability for Kubernetes By Sebastian Wicki, Isovalent
  7. Where is cgroup eBPF? Source: KubeCon2020 - Hubble eBPF Based

    Observability for Kubernetes By Sebastian Wicki, Isovalent
  8. Where is cgroup eBPF? Source: KubeCon2020 - Hubble eBPF Based

    Observability for Kubernetes By Sebastian Wicki, Isovalent
  9. kube-api-server Cilium Agent (daemon_main.go) BPF Maps Watch k8s services, 


    endpoints … update /bpf/init.sh BPFs lookup update k8s Node Cilium Overview Kernel Processe syscall
  10. kube-api-server Cilium Agent (daemon_main.go) BPF Maps Watch k8s services, 


    endpoints … update /bpf/init.sh BPFs lookup update k8s Node Cilium Overview Kernel Processe syscall
  11. Source: https://github.com/cilium/cilium/blob/master/bpf/lib/lb.h Cilium BPF Map LB4_SERVICES_MAP_V2 key value address dport

    backend_slot scope backend_id /
 affinity_timeout count rev_nat index flags flags2 10.10.10.10 80 0 0 0 3 1 10.10.10.10 80 1 0 1 0 1 10.10.10.10 80 2 0 2 0 1 10.10.10.10 80 3 0 3 0 1 Example of a k8s service with 3 backend pods in the 
 LB4_SERVICES_MAP_V2
  12. kube-api-server Cilium Agent (daemon_main.go) BPF Maps Watch k8s services, 


    endpoints … update /bpf/init.sh BPFs lookup update k8s Node Cilium Overview Kernel Processe syscall
  13. Source: Virtual bpfconf 2020 - Alexei Starovoitov, Daniel Borkmann (LSF/MM/BPF

    2020) https://docs.google.com/presentation/d/1w2zlpGWV7JUhHYd37El_AUZzyUNSvDfktrF5MJ5G8Bs/edit#slide=id.g746fc02b5b_3_33 How does Cilium replace kube-proxy?
  14. Source: Virtual bpfconf 2020 - Alexei Starovoitov, Daniel Borkmann (LSF/MM/BPF

    2020) https://docs.google.com/presentation/d/1w2zlpGWV7JUhHYd37El_AUZzyUNSvDfktrF5MJ5G8Bs/edit#slide=id.g746fc02b5b_3_33 How does Cilium replace kube-proxy?
  15. • cgroup eBPF Introduction • commit history • connect syscall

    example • Cilium Agent Overview • LB4_SERVICES_MAP_V2 preparation • Cilium kube-proxy replacement (application side) • NAT on per connect/getpeername/sendmsg/recvmsg syscall, not on per packet Recap