Share
Embed
DevConf.CZ 2026 - vRouter-Operator: Bringing GitOps and IaC to Virtual Network Functions in Kubernetes
DevConf.CZ 2026 - vRouter-Operator: Bringing GitOps and IaC to Virtual Network Functions in Kubernetes
Date Huang
June 20, 2026
Transcript
47 slides
-
vRouter-Operator: Bringing GitOps and IaC to Virtual Network Functions in
Kubernetes Date (Yu-Chiang) Huang <tjjh89017 [at] hotmail.com> @DevConf.CZ 2026-Jun-19 -
Who am I: Date (Yu-Chiang) Huang • Cloud and Network
Solution Architect with 7+ years of experience • Creator of vRouter-Operator, STUNMESH-go and EZIO Project • Expertise in major public cloud networking services and on-prem datacenter networking design • Specialized in Cloud Network, OpenStack, Kubernetes, SD-WAN, and open-source • Extensive speaking experience at international conferences 2 -
Agenda • The Problem • IaC and GitOps • vRouter-Operator
• vRouter-Operator Current Status • Future work • Q&A 3 -
Why This Talk Matters vRouter-Operator turns Virtual Router configuration into
Kubernetes CRDs and delivers it to router VMs without SSH or a management network. 4 -
Network infrastructure today • Virtual Routers run as VMs that
stay on for a long time • The config lives inside the VM, in CLI files or vendor formats • You change things with SSH or the CLI/Console • Apps moved to Kubernetes. The network did not. • Two different tools. Two different teams. Two ways of working. 6 -
How we manage network VMs • Terraform ◦ Manage the
VM instance itself only • Ansible ◦ Sends config over SSH ◦ Not aware of manual changes • Vendor CLI / UI ◦ Manual clicks ◦ No audit trail ◦ Different for each device. Does not scale well 7 -
How we manage network VMs • You need a management
network to reach the VM from. 8 -
The management network tax • Each VM needs an extra
network port so tools can reach it • That port must connect back to a special operator network • You need VLANs, firewall rules, and jump hosts just to run Ansible • Even with VRF isolation, forgetting to bind SSH to the management VRF may expose it to all interfaces 9 -
For multi-tenant network functions • You want to give each
tenant their own router VM • You do not want customers to have a path into your management network 10 -
Configuration drift • Someone logs in by SSH or console
to fix one small thing at 2am • That change is never saved back to the repo • Months later, Ansible runs and breaks production • There is no single source of truth (SSOT). ◦ The VM? ◦ The playbook? ◦ Human memory? 11 -
"Who changed the BGP config last Tuesday? Why" • VM
logs, if they still exist • Of course, you have AAA, but which one is the latest? ◦ Config in the VM? ◦ Config in the script? ◦ Config in the Git repo? • Ansible history, if and only if everyone used it every time • VyOS can roll back config, but which version is the correct one? • This is very different from how app teams work 12 -
Infrastructure as Code • Infrastructure is described in the codes/files
• The files live in version control • If you apply the same files, you get the same result every time • You can review changes with diffs, comments, and approvals 14 -
GitOps • Git holds the desired state of everything •
A controller keeps the cluster in that state all the time • Every change is a pull request. ◦ You can review and revert it. • Cluster-resource drift can be detected and resynced automatically • No one runs commands in production ◦ They merge PRs 15 -
GitOps for apps, Not for the network App in Kubernetes
• Declarative YAML • ArgoCD/Flux • PR review • Automatic rollback • Drift detection • One audit log Network VMs • Ad-hoc CLI • Scattered playbooks • Manual changes • Manual snapshots • Drift hidden • No read audit trail 16 -
So what if… …to manage a router VM the same
way you manage a Deployment with YAML? kubectl apply, pull request, ArgoCD sync, one commit to revert 17 -
What if vRouter/VyOS config was a Kubernetes CR? • Write
router config in CRDs • A controller keeps the vRouter/VyOS VM in sync with the CRD all the time • Now you can use Git, kubectl, ArgoCD, Flux, and OPA • We only changed how the config is written and delivered 18 -
What if putting config to vRouter without mgmt net? •
If we don’t have management network and SSH • Or we don’t want to connect management network with the service/customer networks for security reason • Or bootstrap vRouter VM without any initial configuration for external access ◦ No Cloud-init ◦ No pre-built VM image with initial configuration for external access 19 -
We want IaC/GitOps Friendly Form Need a plaintext form for
the Git and Review. Template Render Engine Reuse the template for different VMs. Reconcile Controller Need a Controller to check the CRDs and push the config periodically. w/o Management Network or SSH Don’t want to enable management network or SSH for security reason. 21 -
Kubernetes CRD for Configuration • Provide IaC/GitOps friendly form to
store the desired state • YAML-based • Store VyOS configuration and commands for execution 22 -
VRouterConfig apiVersion: vrouter.kojuro.date/v1 kind: VRouterConfig metadata: name: my-router-config namespace: default
spec: targetRef: name: vyos-kubevirt config: | interfaces { ethernet eth0 { address dhcp } } ...omitted others commands: | set system host-name 'my-vyos-router-in-kubevirt' 23 -
VRouterTarget apiVersion: vrouter.kojuro.date/v1 kind: VRouterTarget metadata: name: vyos-kubevirt namespace: default
spec: provider: type: kubevirt kubevirt: name: vyos-kubevirt # VirtualMachine name namespace: default params: hostname: "my-vyos-router-in-kubevirt" 24 -
Template Render Engine • Several vRouter VMs with the similar
configurations ◦ e.g. We have BGP-EVPN Spine and Leaf, they have similar configurations but some different neighbors • Different but reusable functions with templates for quick construct ◦ e.g. Same login auth section ◦ Same BGP EVPN section • It will generate the actual config and VRouterConfig • Great for dynamic network service workload, dynamic tenant network provisioning ◦ e.g. Your Customers need a VPN service? Just generate a CRD from template and deploy it 25 -
VRouterTemplate - Hostname apiVersion: vrouter.kojuro.date/v1 kind: VRouterTemplate metadata: name: hostname-template
namespace: default spec: commands: | set system host-name '{{ .hostname }}' 26 -
VRouterTemplate - Interface apiVersion: vrouter.kojuro.date/v1 kind: VRouterTemplate metadata: name: hostname-template
namespace: default spec: commands: | set interface ethernet eth0 address dhcp 27 -
VRouterBinding apiVersion: vrouter.kojuro.date/v1 kind: VRouterBinding metadata: name: config-binding namespace: default
spec: templateRefs: - name: hostname-template - name: interface-template targetRefs: - name: vyos-kubevirt - name: vyos-proxmox 28 -
VRouterTarget apiVersion: vrouter.kojuro.date/v1 kind: VRouterTarget metadata: name: vyos-kubevirt namespace: default
spec: provider: type: kubevirt kubevirt: name: vyos-kubevirt # VirtualMachine name namespace: default params: hostname: "my-vyos-router-in-kubevirt" 29 -
VRouterConfig apiVersion: vrouter.kojuro.date/v1 kind: VRouterConfig metadata: name: config-binding.vyos-kubevirt namespace: default
spec: targetRef: name: vyos-kubevirt commands: | set system host-name 'my-vyos-router-in-kubevirt' set interface ethernet eth0 address dhcp 30 -
The reconciliation loop • The operator watches CRD changes •
When something changes, it reconciles: no polling, no state comparison in the controller side CRD change kubectl / GitOps Reconcile render config vRouter/VyOS config in sync triggers apply 31 -
But, how to push config? • Enable SSH and Push
config via SSH? • Do we still need management network and SSH? 32 -
What if we can remove mgmt net from the vRouter?
• Without SSH • Without Management Network • Without Manual Login Auth for initial configuration for external access 33 -
QEMU Guest Agent (QGA) • QGA is a standard QEMU
feature ◦ The host talks to the VM over a virtio channel - request-response only; the guest cannot initiate • No IP address needed • No SSH session • No open management port inside the VM • VyOS already includes qemu-guest-agent. No custom image needed 34 -
vRouter-Operator Push Config via QGA • vRouter-Operator pushes config to
VyOS VMs in KubeVirt via QGA • Even the VM doesn’t have initial config to allow external access. • Even you mess up your router, you still can push a new config to rollback the system ◦ e.g. If you remove your all network configs accidentally, vRouter-Operator can roll it back! • This works the same on KubeVirt and on Proxmox VE 35 -
KubeVirt VirtualMachine VyOS VM QEMU-GA Libvirt virsh vRouter-Operator VRouterConfig spec:
targetRef: name: vyos-kubevirt commands: | set system host-name '...' set interface ... ... ... ... <omitted other> ... 1. Watch CRDs change 2. Push config QGA 3. Apply Config Change 36 -
IaC/GitOps for Network VMs • Router VM instance is YAML
(KubeVirt) • Router Config is YAML (VRouterConfig) • Everything is IaC/GitOps aware! • Without management network or SSH! 37 -
NOS Supported • VyOS Rolling release • VyOS Stream release
• VyOS 1.5 • (Will support other NOS in the future) 39 -
vRouter-Agent • Beta in v1.2.0, under testing • Need management
network • With gRPC bi-direction, vRouter-Operator can push configs with stream connection • Still don’t need access from external controller via SSH or API • Cloud Controller style to control everything in your edge router or SD-WAN 42 -
Future work - Maybe? • YANG/NETCONF via QGA or Agent
• Other NOS ◦ SONiC, maybe? 44 -
Fully Open Source - Apache 2.0 • https://github.com/tjjh89017/vrouter-operator • Demo
video: https://youtu.be/RsieH9gFU4I ◦ You can find the video in the repo README.md • Contributions welcome! Github 45 -
Questions? • Try it • break it • file an
issue • open a PR • or contact me on LinkedIn. Linkedin Github 47