Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloud Infrastructure Interconnect with Wireguar...

Date Huang
September 01, 2021

Cloud Infrastructure Interconnect with Wireguard and OSPF

Date Huang

September 01, 2021
Tweet

More Decks by Date Huang

Other Decks in Programming

Transcript

  1. About me Date Huang 黃宇強 <tjjh89017 [at] hotmail.com> • 2019

    OpenInfra Day Taiwan Speaker ◦ Massive Bare-Metal Operating System Provisioning Improvement • 2019 OSC Tokyo Fall Speaker • 2019 COScon ‘19 Speaker • 2019 Hong Kong Open Source Conference Speaker ◦ De-centralized Bare-Metal Operating System Provisioning • 2018 ISC High Performance Project Poster Demo ◦ The Design and Implementation of Bare Metal Cluster Deployment Using BitTorrent • 2017 Open Source Summit North America co-Speaker ◦ Building Cloud Infra using cost-effective ARM Boards • 2017 OpenStack Day Taiwan Speaker ◦ Combine Continuous Integration (CI) with OpenStack • 2016 OpenStack Day Taiwan Invited Speaker ◦ OpenStack on ARM64 Open Source Porject • EZIO • STUNMESH-go 2
  2. Overview • Cloud Interconnect Solution Brief ◦ DWDM Solution Brief

    ◦ MPLS Solution Brief ◦ SD-WAN IPsec Brief • Wireguard with OSPF ◦ Wireguard brief ◦ OSPF brief • Example Topology • STUNMESH-go ◦ Wireguard Helper Tool • More than L3 Tunnel 3
  3. DWDM Solution brief - (1) • Dense Wavelength Division Multiplexing

    (DWDM) • Fiber-Optic Communications • Distance: 40, 80, 120KM or more • Merge multiple signal into single paired fiber 5
  4. DWDM Solution brief - (2) • Rent Dark Fiber between

    different location • Connect each others with DWDM Solution 6
  5. DWDM Solution brief - (3) • Pros ◦ Large Bandwidth

    ◦ Full control • Cons ◦ Very Expensive ◦ Very Complex Configuration ◦ Hard to Use and Maintain 7
  6. MPLS VPN brief - (1) • Multi-Protocol Label Switching (MPLS)

    • Using “Label” to determine the route • Provided by ISP 8
  7. MPLS VPN brief - (2) • Pros ◦ Convenience ◦

    Easy to Use and Maintain • Cons ◦ Expensive ◦ Less Bandwidth 9
  8. SD-WAN IPsec brief - (1) • Connect all sites with

    IPsec tunnel via Broadband or Mobile Network • Rich Redundancy between multiple WAN type to Internet ◦ Primary: Broadband Network ◦ Backup: Mobile Network • Need Fixed Public IP in most of time ◦ e.g. AWS VPC Customer Gateway (IPsec) need Public IPs in both side 10
  9. SD-WAN IPsec brief - (2) • Pros ◦ Simple ◦

    Cheap for larger bandwidth ◦ Convenience ◦ Easy to Use and Maintain • Cons ◦ Need Fixed Public IP at Centralized Site ◦ Centralized Architecture 11
  10. What is Wireguard • Simple, Fast, Modern, Secure Tunnel •

    Fast without any hardware acceleration • Support Windows, Linux, MacOS, Android, iOS • Algo ◦ Curve25519 ◦ ChaCha20 ◦ Poly1305 13
  11. Why Wireguard • UDP based VPN Protocol ◦ NAT and

    Firewall Traversal Persistence • Built-in Keepalive • Built-in Roaming ◦ Auto adjust remote peer connection info • Performance better than OpenVPN and IPsec (AES) ◦ Wireguard have good performance without hardware crypto engine ◦ Suitable to Embedded system or Network Box • Much Simpler Configuration than OpenVPN ◦ OpenVPN need 5KB config file ◦ Wireguard only need several bytes • Encapsulate IPv6-in-IPv4, IPv4-in-IPv6 14
  12. What is OSPF • Dynamic Route ◦ Don’t need to

    bother to setup static route when new peer is added • Built-in Keepalive ◦ Fast re-route to redundant route when link status or route is changed • Fast Convergence in small scale network 15
  13. Why OSPF • Built-in Keepalive ◦ Wireguard didn’t have link

    status ◦ It will need to send a packet to remote peer and check if receving reply to check link status ◦ OSPF Hello Packet could test link status and check remote OSPF routing engine status 16
  14. Wireguard with OSPF • Fast and Simple VPN tunnel •

    NAT Traversal Persistence • Dynamic Route in Full Mesh Topology • Fast Re-route • Auto Check Link and Route Status 17
  15. 19

  16. 20

  17. 21

  18. 22

  19. 23

  20. 24

  21. 25

  22. 26

  23. Mobile Network • Through CGNAT ◦ Full Cone NAT ◦

    Translate into Same Port and IP mapping when Destation IP and Port are different • NAT Session to allow ingress traffic ◦ Record Src IP, Src Port, Dst IP, Dst Port ◦ Allow Firewall Rule 28
  24. 29 No Session From Site B to Site A Deny

    Site A to Site B Even Site A know NAT mapping about Site B It still cannot connect to Site B
  25. UDP Hole Punching • Let two clients exchange connection info

    and try to connect to each other via 3rd party server • STUN: Session Traversal Utilities for NAT • STUN is common implement for UDP Hole Punching (RFC 5389) 30
  26. STUNMESH-go • Wireguard helper tool to get through Full-Cone NAT

    ◦ IP and Port translation mapping will be same even different destination • Written in Golang • Multiple Architecture Support ◦ X86_64, MIPS • Standalone Executable ◦ Don’t need to care about library dependency • Inspired by wireguard-p2p project • Open Source ◦ https://github.com/tjjh89017/stunmesh-go ◦ GPLv2 or later • Tested with ◦ UBNT ER-X v2.0.8-hotfix.1 and Wireguard v1.0.20210424 ◦ VyOS 1.4-rolling-202105200417 32
  27. STUNMESH-go • Get Public IP and Port after CG-NAT translation

    ◦ cBPF Filter to receive packet from same UDP port as Wireguard used ◦ Raw Socket to construct STUN 5389 request packet to same UDP port • Encrypt Public Info with Wireguard Curve25519 Key • Save Ciphertext into Cloudflare TXT Record • Query TXT Record from Cloudflare • Decrypt Ciphertext and Update Wireguard Peer Endpoint • Usually, Only need to run once when initiating connection in first time ◦ Or disconnect in the same time 33
  28. 35

  29. 36

  30. 37

  31. 38

  32. 39

  33. 40

  34. 41

  35. 42

  36. 43

  37. 44

  38. 45

  39. 46

  40. 47

  41. More than L3 Tunnel • Wireguard only encapsulate L3 packet

    into tunnel ◦ Start from IP header ◦ Without L2 MAC address, VLAN • Some Protocols or Operations will need L2 ◦ VM migration, two sites need to be in same L2 if VMs connect to each other with same subnet IPs 49
  42. More than L3 Tunnel - L2 Tunnel • VXLAN •

    NVGRE • HARD to control Broadcast, Unknown Unicast, Multicast in L2 tunnel ◦ Proxy ARP ◦ Static MAC table ◦ Static ARP table ◦ BGP-EVPN 50
  43. Reference • https://en.wikipedia.org/wiki/Wavelength-division_multiplexing • https://docs.vmware.com/en/VMware-Smart-Assurance/10.1.0/mpls-man ager-user-guide-101/GUID-8EB1D677-B262-475F-9C1B-8D2D9826CC0D.ht ml • https://www.wireguard.com/ •

    https://zh.wikipedia.org/zh-tw/WireGuard • https://github.com/tjjh89017/stunmesh-go • https://github.com/manuels/wireguard-p2p • https://bford.info/pub/net/p2pnat/ 52