DevConf.CZ 2026 - vRouter-Operator: Bringing GitOps and IaC to Virtual Network Functions in Kubernetes

DevConf.CZ 2026 - vRouter-Operator: Bringing GitOps and IaC to Virtual Network Functions in Kubernetes

Transcript

1. vRouter-Operator: Bringing GitOps and IaC to Virtual Network Functions in Kubernetes Date (Yu-Chiang) Huang <tjjh89017 [at] hotmail.com> @DevConf.CZ 2026-Jun-19
2. Who am I: Date (Yu-Chiang) Huang • Cloud and Network Solution Architect with 7+ years of experience • Creator of vRouter-Operator, STUNMESH-go and EZIO Project • Expertise in major public cloud networking services and on-prem datacenter networking design • Specialized in Cloud Network, OpenStack, Kubernetes, SD-WAN, and open-source • Extensive speaking experience at international conferences 2
3. Agenda • The Problem • IaC and GitOps • vRouter-Operator • vRouter-Operator Current Status • Future work • Q&A 3
4. Why This Talk Matters vRouter-Operator turns Virtual Router configuration into Kubernetes CRDs and delivers it to router VMs without SSH or a management network. 4
5. The Problem 5
6. Network infrastructure today • Virtual Routers run as VMs that stay on for a long time • The config lives inside the VM, in CLI files or vendor formats • You change things with SSH or the CLI/Console • Apps moved to Kubernetes. The network did not. • Two different tools. Two different teams. Two ways of working. 6
7. How we manage network VMs • Terraform ◦ Manage the VM instance itself only • Ansible ◦ Sends config over SSH ◦ Not aware of manual changes • Vendor CLI / UI ◦ Manual clicks ◦ No audit trail ◦ Different for each device. Does not scale well 7
8. How we manage network VMs • You need a management network to reach the VM from. 8
9. The management network tax • Each VM needs an extra network port so tools can reach it • That port must connect back to a special operator network • You need VLANs, firewall rules, and jump hosts just to run Ansible • Even with VRF isolation, forgetting to bind SSH to the management VRF may expose it to all interfaces 9
10. For multi-tenant network functions • You want to give each tenant their own router VM • You do not want customers to have a path into your management network 10
11. Configuration drift • Someone logs in by SSH or console to fix one small thing at 2am • That change is never saved back to the repo • Months later, Ansible runs and breaks production • There is no single source of truth (SSOT). ◦ The VM? ◦ The playbook? ◦ Human memory? 11
12. "Who changed the BGP config last Tuesday? Why" • VM logs, if they still exist • Of course, you have AAA, but which one is the latest? ◦ Config in the VM? ◦ Config in the script? ◦ Config in the Git repo? • Ansible history, if and only if everyone used it every time • VyOS can roll back config, but which version is the correct one? • This is very different from how app teams work 12
13. Infrastructure as Code (IaC) & GitOps 13
14. Infrastructure as Code • Infrastructure is described in the codes/files • The files live in version control • If you apply the same files, you get the same result every time • You can review changes with diffs, comments, and approvals 14
15. GitOps • Git holds the desired state of everything • A controller keeps the cluster in that state all the time • Every change is a pull request. ◦ You can review and revert it. • Cluster-resource drift can be detected and resynced automatically • No one runs commands in production ◦ They merge PRs 15
16. GitOps for apps, Not for the network App in Kubernetes • Declarative YAML • ArgoCD/Flux • PR review • Automatic rollback • Drift detection • One audit log Network VMs • Ad-hoc CLI • Scattered playbooks • Manual changes • Manual snapshots • Drift hidden • No read audit trail 16
17. So what if… …to manage a router VM the same way you manage a Deployment with YAML? kubectl apply, pull request, ArgoCD sync, one commit to revert 17
18. What if vRouter/VyOS config was a Kubernetes CR? • Write router config in CRDs • A controller keeps the vRouter/VyOS VM in sync with the CRD all the time • Now you can use Git, kubectl, ArgoCD, Flux, and OPA • We only changed how the config is written and delivered 18
19. What if putting config to vRouter without mgmt net? • If we don’t have management network and SSH • Or we don’t want to connect management network with the service/customer networks for security reason • Or bootstrap vRouter VM without any initial configuration for external access ◦ No Cloud-init ◦ No pre-built VM image with initial configuration for external access 19
20. vRouter-Operator 20
21. We want IaC/GitOps Friendly Form Need a plaintext form for the Git and Review. Template Render Engine Reuse the template for different VMs. Reconcile Controller Need a Controller to check the CRDs and push the config periodically. w/o Management Network or SSH Don’t want to enable management network or SSH for security reason. 21
22. Kubernetes CRD for Configuration • Provide IaC/GitOps friendly form to store the desired state • YAML-based • Store VyOS configuration and commands for execution 22
23. VRouterConfig apiVersion: vrouter.kojuro.date/v1 kind: VRouterConfig metadata: name: my-router-config namespace: default spec: targetRef: name: vyos-kubevirt config: | interfaces { ethernet eth0 { address dhcp } } ...omitted others commands: | set system host-name 'my-vyos-router-in-kubevirt' 23
24. VRouterTarget apiVersion: vrouter.kojuro.date/v1 kind: VRouterTarget metadata: name: vyos-kubevirt namespace: default spec: provider: type: kubevirt kubevirt: name: vyos-kubevirt # VirtualMachine name namespace: default params: hostname: "my-vyos-router-in-kubevirt" 24
25. Template Render Engine • Several vRouter VMs with the similar configurations ◦ e.g. We have BGP-EVPN Spine and Leaf, they have similar configurations but some different neighbors • Different but reusable functions with templates for quick construct ◦ e.g. Same login auth section ◦ Same BGP EVPN section • It will generate the actual config and VRouterConfig • Great for dynamic network service workload, dynamic tenant network provisioning ◦ e.g. Your Customers need a VPN service? Just generate a CRD from template and deploy it 25
26. VRouterTemplate - Hostname apiVersion: vrouter.kojuro.date/v1 kind: VRouterTemplate metadata: name: hostname-template namespace: default spec: commands: | set system host-name '{{ .hostname }}' 26
27. VRouterTemplate - Interface apiVersion: vrouter.kojuro.date/v1 kind: VRouterTemplate metadata: name: hostname-template namespace: default spec: commands: | set interface ethernet eth0 address dhcp 27
28. VRouterBinding apiVersion: vrouter.kojuro.date/v1 kind: VRouterBinding metadata: name: config-binding namespace: default spec: templateRefs: - name: hostname-template - name: interface-template targetRefs: - name: vyos-kubevirt - name: vyos-proxmox 28
29. VRouterTarget apiVersion: vrouter.kojuro.date/v1 kind: VRouterTarget metadata: name: vyos-kubevirt namespace: default spec: provider: type: kubevirt kubevirt: name: vyos-kubevirt # VirtualMachine name namespace: default params: hostname: "my-vyos-router-in-kubevirt" 29
30. VRouterConfig apiVersion: vrouter.kojuro.date/v1 kind: VRouterConfig metadata: name: config-binding.vyos-kubevirt namespace: default spec: targetRef: name: vyos-kubevirt commands: | set system host-name 'my-vyos-router-in-kubevirt' set interface ethernet eth0 address dhcp 30
31. The reconciliation loop • The operator watches CRD changes • When something changes, it reconciles: no polling, no state comparison in the controller side CRD change kubectl / GitOps Reconcile render config vRouter/VyOS config in sync triggers apply 31
32. But, how to push config? • Enable SSH and Push config via SSH? • Do we still need management network and SSH? 32
33. What if we can remove mgmt net from the vRouter? • Without SSH • Without Management Network • Without Manual Login Auth for initial configuration for external access 33
34. QEMU Guest Agent (QGA) • QGA is a standard QEMU feature ◦ The host talks to the VM over a virtio channel - request-response only; the guest cannot initiate • No IP address needed • No SSH session • No open management port inside the VM • VyOS already includes qemu-guest-agent. No custom image needed 34
35. vRouter-Operator Push Config via QGA • vRouter-Operator pushes config to VyOS VMs in KubeVirt via QGA • Even the VM doesn’t have initial config to allow external access. • Even you mess up your router, you still can push a new config to rollback the system ◦ e.g. If you remove your all network configs accidentally, vRouter-Operator can roll it back! • This works the same on KubeVirt and on Proxmox VE 35
36. KubeVirt VirtualMachine VyOS VM QEMU-GA Libvirt virsh vRouter-Operator VRouterConfig spec: targetRef: name: vyos-kubevirt commands: | set system host-name '...' set interface ... ... ... ... <omitted other> ... 1. Watch CRDs change 2. Push config QGA 3. Apply Config Change 36
37. IaC/GitOps for Network VMs • Router VM instance is YAML (KubeVirt) • Router Config is YAML (VRouterConfig) • Everything is IaC/GitOps aware! • Without management network or SSH! 37
38. vRouter-Operator Current Status 38
39. NOS Supported • VyOS Rolling release • VyOS Stream release • VyOS 1.5 • (Will support other NOS in the future) 39
40. Underlay Providers • KubeVirt • Proxmox VE 40
41. How about Bare-Metal? 41
42. vRouter-Agent • Beta in v1.2.0, under testing • Need management network • With gRPC bi-direction, vRouter-Operator can push configs with stream connection • Still don’t need access from external controller via SSH or API • Cloud Controller style to control everything in your edge router or SD-WAN 42
43. Future work 43
44. Future work - Maybe? • YANG/NETCONF via QGA or Agent • Other NOS ◦ SONiC, maybe? 44
45. Fully Open Source - Apache 2.0 • https://github.com/tjjh89017/vrouter-operator • Demo video: https://youtu.be/RsieH9gFU4I ◦ You can find the video in the repo README.md • Contributions welcome! Github 45
46. Demo 46
47. Questions? • Try it • break it • file an issue • open a PR • or contact me on LinkedIn. Linkedin Github 47