Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
仮想化環境での利用者公平性
Search
Takuya ASADA
November 20, 2012
Technology
0
170
仮想化環境での利用者公平性
Takuya ASADA
November 20, 2012
Tweet
Share
More Decks by Takuya ASADA
See All by Takuya ASADA
Interrupt Affinityについて
syuu1228
0
270
僕のIntel NUCが起動しないわけがない
syuu1228
3
4.4k
Introduction to bhyve
syuu1228
1
390
OSv on bhyve
syuu1228
3
400
ruby-virtualmachine
syuu1228
0
260
BHyVeでOSvを起動したい 〜BIOSがなくてもこの先生きのこるには〜
syuu1228
2
840
10GbE時代のネットワークI/O高速化
syuu1228
14
8.7k
Play with UEFI
syuu1228
1
330
/proc/irq/<IRQ>/ smp_affinity
syuu1228
0
430
Other Decks in Technology
See All in Technology
OAuth/OpenID Connectで実現するMCPのセキュアなアクセス管理
kuralab
5
840
変化する開発、進化する体系時代に適応するソフトウェアエンジニアの知識と考え方(JaSST'25 Kansai)
mizunori
0
130
Create a Rails8 responsive app with Gemini and RubyLLM
palladius
0
140
LinkX_GitHubを基点にした_AI時代のプロジェクトマネジメント.pdf
iotcomjpadmin
0
160
AIの最新技術&テーマをつまんで紹介&フリートークするシリーズ #1 量子機械学習の入門
tkhresk
0
120
BigQuery Remote FunctionでLooker Studioをインタラクティブ化
cuebic9bic
2
220
VCpp Link and Library - C++ breaktime 2025 Summer
harukasao
0
220
第9回情シス転職ミートアップ_テックタッチ株式会社
forester3003
0
130
_第3回__AIxIoTビジネス共創ラボ紹介資料_20250617.pdf
iotcomjpadmin
0
140
Clineを含めたAIエージェントを 大規模組織に導入し、投資対効果を考える / Introducing AI agents into your organization
i35_267
4
1.4k
実践! AIエージェント導入記
1mono2prod
0
140
強化されたAmazon Location Serviceによる新機能と開発者体験
dayjournal
2
150
Featured
See All Featured
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
920
Code Review Best Practice
trishagee
68
18k
Building Adaptive Systems
keathley
43
2.6k
Docker and Python
trallard
44
3.4k
Writing Fast Ruby
sferik
628
61k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
16
940
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
8
660
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
How to Ace a Technical Interview
jacobian
277
23k
Transcript
ԾԽڥͰͷ ར༻ऀެฏੑ ઙా @syuu1228 1211݄20Ր༵
ԾԽڥͷެฏੑʁ • KVMͷήετΛͨ͘͞Μ্ཱͪ͛ͨ࣌ɺϊʔυؒ ͷ௨৴ੑೳCPUར༻࣌ؒͷެฏੑͲͷఔอ ͨΕ͍ͯΔͷ͔ • CPUར༻࣌ؒ • ௨৴ྔ •
ϨΠςϯγ • VMลΓͷෛՙͷΒ͖ͭ 1211݄20Ր༵
ଌͬͯΈΑ͏ • ϨΠςϯγɾ௨৴ྔɾCPU༻ͳͲͷ໘Ͱ Ͳͷఔެฏͳεέδϡʔϧ͕ߦΘΕΔͷ͔ • VMΛ࣮CPUҎ্ʹ૿ͨ࣌͠ʹԿ͕ى ͖Δ͔ • ԾNICͱSR−IOVͰੑೳಛੑ͕ͲͷΑ͏ ʹҟͳΔ͔
1211݄20Ր༵
ੑೳଌఆڥ • ̎ͷLinuxػɾ10G NIC • 1ʙ64ͷKVMήετɾ128ϓϩηεͷ netperf 1VM→VM͋ͨΓ128ϑϩʔ 2VM→VM͋ͨΓ64ϑϩʔ 4VM→VM͋ͨΓ32ϑϩʔ…
64VM→VM͋ͨΓ2ϑϩʔ • TCP Request/ResponseϞʔυ 1byteͷύέοτΛϐϯϙϯ VMϗετ ήετ ςετػ 10G NIC 10G NIC ήετ netperf netserver netperf 1211݄20Ր༵
ϋʔυʗιϑτͷεϖοΫ Distribution Ubuntu Server 12.10 Linux Kernel 3.5.0-18-generic QEMU-KVM 1.2.0
Netperf 2.5.0 CPU(VMϗετ) Intel Core i7 980 (3.33GHz) Memory(VMϗετ) 24GB CPU(ςετػ) Intel Core i7 860 (2.8GHz) Memory(ςετػ) 8GB NIC Intel 82599(ixgbe) ཧ6ίΞɺཧ12ίΞ 1211݄20Ր༵
ήετϚγϯͷઃఆ • ԾCPUɿ1ͭ • ԾNICͷઃఆ • macvtapͰbridge / vhost-net •
SR-IOV • 512MBͷϝϞϦׂΓͯ 1211݄20Ր༵
͓͞Β͍ɿSR-IOV • ཧNIC͕VMʹରͯ͠ԾNICΛఏڙɹIOϋΠύʔόΠβʔΛհࡏͤͣʹߦΘΕΔ • ׂࠐΈ͚ͩԾԽग़དྷ͍ͯͳ͍ͷͰKVMΛ௨ͯ͡సૹ͞ΕΔ • ࠷ੑೳ͕ߴ͍͕ϋʔυରԠ͕ඞཁʗήετͷύέοτΛϑΟϧλʔɾվม͢Δࣄ͍͠ NIC Χʔωϧ Χʔωϧ
ήετ Ϣʔβ PF PFυϥΠό qemu kvm VF υϥΠό TCP/IP ελοΫ NIC2 ϑΥϫʔυ ׂΓࠐΈ ཧׂΓࠐΈ ׂΓࠐΈ ׂࠐΈ ϋϯυϥ VF %."ύεεϧʔ ϋΠύʔόΠβ 1211݄20Ր༵
macvtap <interface type='direct'> <mac address='52:54:00:6b:28:01'/> <source dev='eth1' mode='vepa'/> <model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </interface> 1211݄20Ր༵
SR-IOV <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10'
function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </hostdev> 1211݄20Ր༵
ൺֱରͱͯ͠ͷ ࣮ػੑೳଌఆ • ̎ͷLinuxػɾ10G NIC • KVMήετͰnetserver࣮ߦ • 128ϓϩηεͷnetperf •
TCP Request/ResponseϞʔυ 1byteͷύέοτΛϐϯϙϯ VMϗετ ςετػ 10G NIC 10G NIC netperf netserver netperf netserver 1211݄20Ր༵
VMΛ1-64·Ͱ૿Ճ vhost-netͱSR-IOVൺֱ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ 0 1375.00 2750.00 4125.00 5500.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM transaction/sec ཧίΞʹ͍ۙลΓͰ ੑೳ࠷ߴ 1211݄20Ր༵
ϨΠςϯγ 0 750.00 1500.00 2250.00 3000.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM ϨΠςϯγ ཧίΞʹ͍ۙลΓͰ ϨΠςϯγ࠷ద 1211݄20Ր༵
CPUෛՙ 0 25.00 50.00 75.00 100.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM ύʔηϯςʔδ ෛՙΛVMͷίΞʹ ͔͠ࢄͰ͖ͯͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 30% 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 30% 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 20% 1211݄20Ր༵
ൺֱ݁Ռ • ཧίΞʹ͍ۙVMͩͱੑೳ͕Ұ൪ྑ͍Α͏ʹ ݟ͑Δ • ཧίΞͷ10ഒͷVMΛ࣮ߦͯ͠ɺ͜ͷෛՙύλ ʔϯͰۃͳϨΠςϯγͷѱԽෛՙͷภΓͳͲ ݟΒΕͳ͔ͬͨ • ཧڥͱൺ͔ͯͳΓੑೳ͕ѱ͍
→ෛՙΛෳίΞࢄग़དྷ͍ͯͳ͍ࣄ͕ݪҼͷީ ิͱͯ͠ߟ͑ΒΕΔ 1211݄20Ր༵
Ծڥ্Ͱ ෳίΞωοτϫʔΫIO ෛՙࢄͤͨ࣌͞ͷੑೳ 1211݄20Ր༵
Ծڥ্ͰίΞࢄʁ • ԾCPUΛ1-16·Ͱ૿ͯ͠ΈΔ • ίΞΛ૿͚ͨͩ͠ͰωοτϫʔΫε λοΫ͕ฒྻͰΒͳ͍ • ✕ RSSɿԾNICɾVF͕ະରԠ •
◦ RPSɿήετଆLinuxͷઃఆͰ༗ޮԽ͠ ͯΈΔ 1211݄20Ր༵
RPS $ echo "f" > /sys/class/net/eth1/queues/rx-0/ rps_cpus $ echo 4096
> /sys/class/net/eth1/queues/ rx-0/rps_flow_cnt $ echo 32768 > /proc/sys/net/core/ rps_sock_flow_entries 1211݄20Ր༵
vCPU <vcpu placement='static'>16</vcpu> 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢvhost-netʣ 0 1375.00 2750.00 4125.00 5500.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢvhost-net, RPSʣ 0 1375.00 2750.00 4125.00 5500.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒRPSͰ ੑೳվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢSR-IOVʣ 0 1375.00 2750.00 4125.00 5500.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢSR-IOV, RPSʣ 0 1375.00 2750.00 4125.00 5500.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒRPSͰ ੑೳվળ 1211݄20Ր༵
ϨΠςϯγ ʢvhost-netʣ 0 750.00 1500.00 2250.00 3000.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢvhost-net, RPSʣ 0 750.00 1500.00 2250.00 3000.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢSR-IOVʣ 0 750.00 1500.00 2250.00 3000.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢSR-IOV, RPSʣ 0 750.00 1500.00 2250.00 3000.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
CPUෛՙ ʢvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VM =< 4ͳΒɺcpu = 1 ͷ࣌ΑΓෛՙΛࢄͰ ͖͍ͯΔʁ 1211݄20Ր༵
CPUෛՙ ʢvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ RPSͰΑΓෛՙΛࢄ 1211݄20Ր༵
CPUෛՙ ʢSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
CPUෛՙ ʢSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ cpu >= 8, VM >=16ͷ࣌ʹ 15%Λ͑ΔภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯ/vhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ cpu >= 8, VM >=16ͷ࣌ʹ 15%Λ͑ΔภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ͋·Γେ͖ͳภ ΓݟΒΕͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ͋·Γେ͖ͳภ ΓݟΒΕͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VMͷ্ঢͰ एׯ૿Ճ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VMͷ্ঢͰ एׯ૿Ճ 1211݄20Ր༵
ൺֱ݁Ռ • ԾϚγϯͰ࣮ػੑೳͷҎ্Λग़ͤͨͷVM͕͘͝গͳ͘ vCPU͕͔ͳΓଟ͍࣌ͷΈ → ͜ͷ݅Ͱ͔͠RPSͰੑೳ͕ग़ͳ͍ • ૯vCPUΛଟͯ͘͠ҙ֎ͱੑೳԼ͕Βͳ͍ • SR-IOVͷ߹ʹVF͕ཧίΞΛ͑ͨลΓͰ͔ͳΓੑೳʹภ
Γ͕ͰΔ • NIC্ͷΩϡʔʹରͯ͠ड৴ॲཧʹׂΓͯΔίΞ͕ෆͯ͠ ͜ͷΑ͏ͳঢ়گʹͳΔͱਪଌ • vhost-netͰͦͷΑ͏ͳݱ͕ݟΒΕͣɺ૯vCPUΛ૿ͯ͠ ൺֱతެฏʹॲཧ͞Ε͍ͯΔΑ͏ʹݟ͑Δ 1211݄20Ր༵
ϚϧνϢʔβ࣌ͷެ ฏੑ 1211݄20Ր༵
ϚϧνϢʔβ࣌ͷެฏੑ • σϑΥϧτεέδϡʔϦϯάͰɺશͯͷKVM ήετϓϩηε͕ެฏʹCPU࣌ؒΛׂΓͯΒ ΕΔ →ෳͷϢʔβ͕ෳͷVMΛىಈ͍ͯ͠Δ ߹ɺϢʔβลΓͷϦιʔεׂ͕ෆެฏʹͳΔ • ϢʔβؒͰެฏʹεέδϡʔϦϯάͯ͠ཉ͍͠ 1211݄20Ր༵
cgroup • ϓϩηεάϧʔϓʹରͯ͠Ϧιʔε ʢCPUɺϝϞϦɺσΟεΫI/Oɺωοτ ϫʔΫI/Oʣͷར༻Λ੍ݶɾִ͢Δҝ ͷLinuxͷػೳ 1211݄20Ր༵
cgroup tool install $ apt-get install cgroup-bin 1211݄20Ր༵
$ ls /sys/fs/cgroup blkioɹϒϩοΫI/Oͷ੍ݶ cpuɹCPUϦιʔεͷ੍ݶ cpuacctɹCPUͷߦ࣌ؒͳͲͷใऩू cpusetɹCPUͷׂΓ͚ʹؔ͢Δ੍ݶ devicesɹσόΠεͷΞΫηεڐՄɾڋ൱ freezerɹάϧʔϓʹଐ͢ΔϓϩηεͷҰ࣌ఀࢭɾ࠶։ memoryɹϝϞϦϦιʔεͷ੍ݶ
net_clsɹૹग़ύέοτʹtc༻ͷλάΛઃఆ nsɹωʔϜεϖʔεʢίϯςφ༻ʣ 1211݄20Ր༵
cd /sys/fs/cgroup/cpu $ mkdir grp_a $ echo ‘12254’ > grp_a/tasks
$ echo ‘2184’ > grp_a/tasks $ echo ‘512’ > grp_a/cpu.shares $ mkdir grp_b $ echo ‘9012’ > grp_b/tasks $ echo ‘1024’ > grp_b/cpu.shares άϧʔϓ࡞ άϧʔϓॴଐϓϩη εΛՃʢPIDʣ άϧʔϓͷλεΫͰ༻ग़དྷ ΔCPU࣌ؒͷ૬ରత 1211݄20Ր༵
libvirtdͱcgroup • /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>ʹ VMؔ࿈ϓϩηε͕ࣗಈొ͞ΕΔ • /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/ vcpuNʹvcpuϓϩηε͕ࣗಈొ͞ΕΔ • libvirtܦ༝ͰcgroupͷϦιʔε੍ݶઃఆ ͕ग़དྷΔ
1211݄20Ր༵
virshͱcgroup $ virsh schedinfo vm0 Scheduler : posix cpu_shares :
1024 vcpu_period : 100000 vcpu_quota : -1 $ virsh schedinfo --set cpu_shares=512 vm0 1211݄20Ր༵
̎ͭͷάϧʔϓʹެฏʹϦ ιʔεΛׂΓͯͯΈΔ • /sys/fs/cgroup/grp_a • VMΠϯελϯεɿ̒ݸ • cpu.sharesɿσϑΥϧτʢ̍̌̎̐ʣ • /sys/fs/cgroup/grp_b
• VMΠϯελϯεɿ̍̌ݸ • cpu.sharesɿσϑΥϧτʢ̍̌̎̐ʣ 1211݄20Ր༵
άϧʔϓͷ࡞Γํ $ virsh start <VM໊> $ mkdir /sys/fs/cgroup/cpu/grp_a $ cat
/sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/tasks > tmp $ cat /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/ vcpu*/tasks >> tmp $ cat tmp > /sys/fs/cgroup/cpu/grp_a/tasks 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢcgroupະ༻࣌ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢcgroup༻࣌ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ grp_a grp_b
1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢcgroupະ༻࣌ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ 1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢcgroup༻࣌ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ grp_a grp_b
1211݄20Ր༵
άϧʔϓ͝ͱͷ߹ܭൺֱ ʢcgroup༻࣌ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.10 1.14 1211݄20Ր༵
ʦࢀߟʧάϧʔϓ͝ͱͷ߹ܭൺֱ ʢcgroupະ༻࣌ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.64 1.65 1211݄20Ր༵
ൺֱ݁Ռ • άϧʔϓʹ͚Δ͚ͩͰσϑΥϧτͰάϧʔϓεέ δϡʔϦϯά͕ߦΘΕɺ࣮ଌͰάϧʔϓʹର͠ ͯެฏʹCPU͕͞Ε͍ͯΔࣄ͕֬ೝͰ͖Δ • CPUΛߜΔࣄͰ݁ՌతʹωοτϫʔΫIOΛߜΔࣄ ͋Δఔग़དྷ͍ͯΔ • Ͱ͋·Γݫີ͡Όͳ͍ʢ࣮ଌ14%͘Β͍ͷޡࠩʣ
1211݄20Ր༵
CFS bandwidth control • ͜ͷάϧʔϓcpu.cfs_period_usͷ୯Ґ࣌ ؒதʹ࠷େcpu.cfs_quota_us͚࣮ͩޮग़དྷΔ • άϧʔϓͷϓϩηεperiod୯Ґ࣌ؒ தɺಉ࣌ʹ࣮ߦ͞Ε͏ΔͷͰquotaperiod Λ͑͏Δʢ2CPUͰΊ͍͍ͬͺ͍ͬͨ
Βquota = period * 2ʣ 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢquota=200000,period=50000ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ grp_a grp_b
1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢquota=200000,period=50000ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ grp_a grp_b
1211݄20Ր༵
άϧʔϓ͝ͱͷ߹ܭൺֱ ʢquota=200000,period=50000ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.01 0.63 1211݄20Ր༵
ݕূ݁Ռ • ͔֬ʹάϥϑ্៉ྷʹͳΔͷ͕ͩɺCPU͕ͲΜͳʹ ۭ͍͍ͯͯquota·ͰߦͬͨΒઈର࣮ߦ͞Εͳ͍ • ͔ͱ͍࣮ͬͯߦ͖͠Εͳ͍ఔͷquotaΛ༩͑Δͱઃఆ͠ ͳ͍߹ͱมΘΒͳ͘ͳΔ • CPUωοτϫʔΫIOͪΐ͏ͲΑ͘ͽͬͨΓ1.0ͱ͍ ͏༁ʹ͍͔ͳ͔ͬͨ
• period͕͗͢ΔͱVM͕͍࣮ؒߦ͞Εͳͯ͘ྑ͘ͳ ͍͜ͱ͕ى͖ͦ͏ 1211݄20Ր༵
net_cls $ tc class add dev virbr0 parent 10: classid
10:1 htb rate 24Mbit $ echo 0x100001 > /sys/fs/cgroup/net_cls/ grp_a/net_cls.classid ϓϩηε͔Βૹ৴͞ΕΔύέοτʹtcͷ λάΛ͚ΔࣄʹΑΓଳҬ੍ݶ͕ग़དྷΔ 1211݄20Ր༵
·ͱΊ • KVMʹΑΔԾԽڥͷϦιʔεׂΓͯͷެฏੑʹ͍ͭͯ ݕূͨ͠ • ࠓճͷ࣮ݧͷ݅ԼͰɺ࣮CPUΛେ͖͑͘ΔVMͰ ۃͳϨΠςϯγͷѱԽෛՙͷภΓͳͲݟΒΕͳ͔ ͬͨ • ୠ͠ɺSR-IOVͷ߹ʹVF͕ཧίΞΛ͑ͨลΓͰ͔
ͳΓੑೳʹภΓ͕ൃੜͨ͠ • cgroupΛ༻͍ͯϚϧνϢʔβ࣌ʹϢʔβؒͷCPUׂΓͯΛ ެฏʹग़དྷΔࣄΛ֬ೝͨ͠ 1211݄20Ր༵