of Tokyo) • Github: @mmisono • A Committer of bpftrace (130+commits) • Attending BitVisor Summit from 2016- • 2017 : BPFΛར༻ͨ͠BitVisor෦ͰͷύέοτϑΟϧλϦϯά (+α) (Packet filtering (+α) in BitVisor with BPF) • 2018 : bitvisor.ko : BitVisor as a module • 2019 : vIOMMU implementation in BitVisor
• Performance evaluation of BitVisor itself is not so easy • VMEXIT count, latency • When shadow driver working • … • How can we get this? • We want to tracing tool for BitVisor!
BPF program • Implement a hypercall (vmcall/vmmcall) to load a BPF program • Implement a hypercall to get a BPF map • Statically instrument tracing points (like Linux’s tracepoint)
specific to BitVisor • We modified BCC to generate dedicated BPF code for BitVisor • BCC (and other tools) are actively developed • Can we use reduce the modification of userland program?
very hard • Our approach • Safe execution by running BPF program In a VMX root ring3 (a.k.a protection domain) VMX ROOT mode Ring0 Ring3 BPF VM BPF map BitVisor Main Thread Protection Domain Helper Functions an event
BitVisor Main Thread Protection Domain Helper Functions an event Ring0 BPF VM BPF map BitVisor Main Thread Helper Functions an event 2017 2020 Messaging overhead, but gives the safety
program can call external functions by BPF CALL instruction • Implement the same helper function • CALL 1 : BPF_MAP_LOOKUP_ELEM • CALL 2 : BPF_MAP_UPDETE_ELEM • CALL 3 : BPF_MAP_DELETE_ELEM • …. BPF VM BPF map Helper Functions Other functions
the before) • ubpf is an userland eBPF VM: https://github.com/iovisor/ubpf • Implement basic BPF helper functions in a protection domain • Modify bpftrace so that it call vmmcall instead of system call when interacting BPF functionality • I do not change any BPF code generation part of bpftrace!
call BPF VM and pass data • The mechanism to communicate with other threads/processes using callbacks Ring0 Ring3 BPF VM BPF map BitVisor Main Thread Protection Domain Helper Functions an event sendmsg()
ubpf and third-party libraries) • bpftrace ~300LOC • This includes comments, blank lines and debug codes. The actual amount of modification is much smaller
OS vmcall handler 2. register the program ① Loading BPF Program bpftrace 1. compile & load BPF program Helper Functions map BitVisor Main Thread Protection Domain
1. Request the map data map 2. Retrieve map 3. return map data ③ Retrieve tracing information Helper Functions bpftrace BitVisor Guest OS BitVisor Main Thread Protection Domain
a message passing? (ring0 ó ring3) • I developed and experimented everything on VMWare Fusion on macOS with nested virtualization • Therefore, no legitimate evaluation presentation today :( • One day I want to try to do proper evaluation
very preliminarily • Only support an integer key/value pair map • I guess it’s not so hard to extend this • What can we do if this project work enough? • Guest-Host cooperating tracing • BitVisor introspection according to the guest behavior • … • I think there must be a lot of fun things to do!
• By utilizing a protection domain, BPF program run with safety guaranteed in a VMX root mode • Extend bpftrace and users can trace BitVisor with it • Let’s enjoy tracing! !