5 • A Software router from scratch • Use DPDK and Poptrie to accelerate packet forwarding and route-lookup • Paper about routing lookup algorithm ◦ H.Asai,Y.Ohara,"Poptrie:A Compressed Trie with Population Count for Fast and Scalable Software IP Routing Table Lookup", ACM SIGCOMM '15, p.57-70. • Patent about routing lookup algo. ◦ PCT/JP2016/052664(2016.01.29) ◦ 特許第5960863号(特願2015-048657)2016/7/1registered • Participated in the Interop Tokyo 2018 (Shownet) as a Core-Router • Next Step: Support High performance NFV..?, More Portability..?
a router on JUST COTS 6 Higher performance NFV • High speed Implementation .. ? • no Open Implementation .. ? • using DPDK .. ? Agility / Extensibility • Special Policy • New Protocols • Automation
Software Router 7 High-speed software router using DPDK and Poptrie. Achieve 100GbE routing with COTS and Software We believe currently (as a software forwarder) its performance is the highest in the world. Usecase • Enterprise Router • High-performance VNF • Load Balancer • etc... 導入先のアイディア等, ご意見を募集しています!
trie for IP routing table lookup 8 Small memory footprint with population count • 2.4MiB for a global tier-1 ISP’s full route Good performance through comprehensive evaluation • 240 Mlps/core (with Tier 1 backbone full route 531K routes, random address) Presented at ACM SIGCOMM’15. H.Asai(U-Tokyo), Y.Ohara(NTT-Com) • NTT-Com have a patent right of an algorithm.
18 • High-speed custom-made software router for the era of 100GbE-capable in software • Targeting to the SDN/NFV • Extensibility (Custom for the Special-purpose box and/or service)
a Router / What do we need from a Router..? 20 • IP Forwarding • Perform Protocols • Packet Filtering • Statistics • Management • Support Protocol • High Performance forwarding • User Friendly
Principle 22 • Data Plane ◦ Run to completion on DPDK space ◦ No memory-copy, context-switch, lock ◦ IP Forwarding / Packet Filtering / etc... • Control Plane ◦ Use Linux Kernel Network-Stack, partially ◦ Use Tap interface to connect between DPDK and Kernel space ◦ BGP / OSPF / ICMP / etc... • Management Plane ◦ Be able to control and manage Kamuee with VTY
Principle 23 • Dataplane’s Basic Task ◦ IP Forwarding / IP Lookup / ACL / Vlan tagging ◦ Parallelism multiple threads using RSS ◦ NUMA aware core consumption • IP Forwarding ◦ Use Poptrie for route-lookup • ACL ◦ Using DPDK’s librte_acl ◦ Provides simple ACL for egress/ingress traffic to the interfaces
RSS for Scaling 24 • Use RSS for traffic balancing • High performance w/ special HW feature • 40GbE : 4 cores/port • 100GbE: 8 cores/port Cores consumption style is based on dpdk/examples/L3fwd
Processing on DPDK 26 DPDK is set of libraries for fast packet processing it uses some mechanizm to bypass Kernel-Network-Stack. (UIO,isolcpu, pthread-setaffinity, special Memory management with hugepages) 1. No Memory Copy 2. No Context Switch 3. Full scratch for perf 4. Avoid Task Scheduler
Processing on Linux Kernel 27 There are some mechanisms makes network-application slow. Because Linux is not developed for the fast packet processing, it is developed for the general purpose use. 1. Memory Copy 2. Context Switch 3. Fat Network Stack 4. Task Scheduler
M-plane Design Principle 28 • Partially offload features to Linux-Kernel ◦ Netlink is key-points ◦ ARP is exceptionally different • Routing/Management Protocols ◦ Use Quagga/net-snmp ◦ Partially offload features to Linux-Kernel ◦ Subscribe Linux-Network-Stack’s update using Netlink
Support Routing Protocols with Fast Forwarding 29 • 2 type Data-path ◦ Slow-path: Redirect to the Kernel via Tap interface ◦ Fast-path: Run to completion in DPDK-space • Linux kernel can perform TCP/IP so-stable
Future works 34 • Linux-FIB’s Limitation ◦ Designed for basic ip forwarding only ◦ ex. can’t perform flowspec (necessary iptables) ◦ Special implementation is needed (as well vlan/arp) • Feature stability is needed ◦ c-plane resource isolation ◦ enhance UI, stats, VRF, ACL ◦ support new protocol • Performance Evaluation ◦ only 2 developer, we want you
• High-speed custom-made software router for the era of 100GbE-capable in software • Targeting to the SDN/NFV • Extensibility (Custom for the Special-purpose box and/or service) • Contact: [email protected]
Algorithm 44 Using RCU enables forwarding thread can access FIB as a lockfree. RCU with QSBR algorithm provides the best bandwidth throughput and frames per second throughput when handling 64 byte packets. ref: Y.Ohara, Y.Yamagishi, Kamuee Zero, IC’2016
a Lightweight Thread 46 Tasks to be needed as a fast • dplane: packet forwarding Tasks to be not needed as a fast • cplane: routing protocol • cplane: UI ただ, 扱いが難しいから今後 pthreadに 移行する可能性あり
55 ref: https://dpdksummit.com/Archive/pdf/2016Germany/DPDK-2016-DPDK_FD_IO_Introduction.pdf DPDK is the Data Plane Development Kit that consists of libraries to accelerate packet processing workloads running on a wide variety of CPU architectures. (by www.dpdk.org)
on Linux Kernel 56 There are some mechanisms makes network-application slow. Because Linux is not developed for the fast packet processing, it is developed for the general purpose use. 1. Memory Copy 2. Context Switch 3. Fat Network Stack 4. Task Scheduler
on DPDK 57 DPDK is set of libraries for fast packet processing it uses some mechanizm to bypass Kernel-Network-Stack. (UIO,isolcpu, pthread-setaffinity, special Memory management with hugepages) 1. No Memory Copy 2. No Context Switch 3. Full scratch for perf 4. Avoid Task Scheduler
slankdev
kuri8ive
masatoto
okdt
daimoriwaki
hargon24
trafficbrain
ryotoitoi
keisuke198619
moda0
upura
akifumi_wachi
jacobian
csswizardry
mongodb
jlugia
shlominoach
brettharned
thoeni
hannesfritz
hatefulcrawdad
geoffreycrofte
aki_iinuma
smashingmag
