Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
仮想化環境での利用者公平性
Search
Takuya ASADA
November 20, 2012
Technology
0
150
仮想化環境での利用者公平性
Takuya ASADA
November 20, 2012
Tweet
Share
More Decks by Takuya ASADA
See All by Takuya ASADA
Interrupt Affinityについて
syuu1228
0
200
僕のIntel NUCが起動しないわけがない
syuu1228
3
4.2k
Introduction to bhyve
syuu1228
1
330
OSv on bhyve
syuu1228
3
370
ruby-virtualmachine
syuu1228
0
230
BHyVeでOSvを起動したい 〜BIOSがなくてもこの先生きのこるには〜
syuu1228
2
800
10GbE時代のネットワークI/O高速化
syuu1228
14
8.5k
Play with UEFI
syuu1228
1
280
/proc/irq/<IRQ>/ smp_affinity
syuu1228
0
370
Other Decks in Technology
See All in Technology
わたしとトラックポイント / TrackPoint tips
masahirokawahara
1
240
現地でMeet Upをやる場合の注意点〜反省点を添えて〜
shotashiratori
0
520
使えそうで使われないCloudHSM
maikamibayashi
0
170
Automated Promptingを目指すその前に / Before we can aim for Automated Prompting
rkaga
0
110
pandasはPolarsに性能面で追いつき追い越せるのか
vaaaaanquish
4
4.6k
Autify Company Deck
autifyhq
1
39k
AWS CDKでデータリストアの運用、どのように設計する?~Aurora・EFSの実践事例を紹介~/aws-cdk-data-restore-aurora-efs
mhrtech
4
650
Commitment vs Harrisonism - Keynote for Scrum Niseko 2024
miholovesq
6
1.1k
Oracle Cloud Infrastructureデータベース・クラウド:各バージョンのサポート期間
oracle4engineer
PRO
27
12k
CyberAgent 生成AI Deep Dive with Amazon Web Services / genai-aws
cyberagentdevelopers
PRO
1
480
よくわからんサービスについての問い合わせが来たときの強い味方 Amazon Q について
kazzpapa3
0
220
Gradle: The Build System That Loves To Hate You
aurimas
2
150
Featured
See All Featured
GraphQLの誤解/rethinking-graphql
sonatard
66
9.9k
Fashionably flexible responsive web design (full day workshop)
malarkey
404
65k
Git: the NoSQL Database
bkeepers
PRO
425
64k
Designing Experiences People Love
moore
138
23k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
28
7.9k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
328
21k
Being A Developer After 40
akosma
86
590k
Side Projects
sachag
452
42k
GitHub's CSS Performance
jonrohan
1030
460k
Building Applications with DynamoDB
mza
90
6.1k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
355
29k
StorybookのUI Testing Handbookを読んだ
zakiyama
26
5.2k
Transcript
ԾԽڥͰͷ ར༻ऀެฏੑ ઙా @syuu1228 1211݄20Ր༵
ԾԽڥͷެฏੑʁ • KVMͷήετΛͨ͘͞Μ্ཱͪ͛ͨ࣌ɺϊʔυؒ ͷ௨৴ੑೳCPUར༻࣌ؒͷެฏੑͲͷఔอ ͨΕ͍ͯΔͷ͔ • CPUར༻࣌ؒ • ௨৴ྔ •
ϨΠςϯγ • VMลΓͷෛՙͷΒ͖ͭ 1211݄20Ր༵
ଌͬͯΈΑ͏ • ϨΠςϯγɾ௨৴ྔɾCPU༻ͳͲͷ໘Ͱ Ͳͷఔެฏͳεέδϡʔϧ͕ߦΘΕΔͷ͔ • VMΛ࣮CPUҎ্ʹ૿ͨ࣌͠ʹԿ͕ى ͖Δ͔ • ԾNICͱSR−IOVͰੑೳಛੑ͕ͲͷΑ͏ ʹҟͳΔ͔
1211݄20Ր༵
ੑೳଌఆڥ • ̎ͷLinuxػɾ10G NIC • 1ʙ64ͷKVMήετɾ128ϓϩηεͷ netperf 1VM→VM͋ͨΓ128ϑϩʔ 2VM→VM͋ͨΓ64ϑϩʔ 4VM→VM͋ͨΓ32ϑϩʔ…
64VM→VM͋ͨΓ2ϑϩʔ • TCP Request/ResponseϞʔυ 1byteͷύέοτΛϐϯϙϯ VMϗετ ήετ ςετػ 10G NIC 10G NIC ήετ netperf netserver netperf 1211݄20Ր༵
ϋʔυʗιϑτͷεϖοΫ Distribution Ubuntu Server 12.10 Linux Kernel 3.5.0-18-generic QEMU-KVM 1.2.0
Netperf 2.5.0 CPU(VMϗετ) Intel Core i7 980 (3.33GHz) Memory(VMϗετ) 24GB CPU(ςετػ) Intel Core i7 860 (2.8GHz) Memory(ςετػ) 8GB NIC Intel 82599(ixgbe) ཧ6ίΞɺཧ12ίΞ 1211݄20Ր༵
ήετϚγϯͷઃఆ • ԾCPUɿ1ͭ • ԾNICͷઃఆ • macvtapͰbridge / vhost-net •
SR-IOV • 512MBͷϝϞϦׂΓͯ 1211݄20Ր༵
͓͞Β͍ɿSR-IOV • ཧNIC͕VMʹରͯ͠ԾNICΛఏڙɹIOϋΠύʔόΠβʔΛհࡏͤͣʹߦΘΕΔ • ׂࠐΈ͚ͩԾԽग़དྷ͍ͯͳ͍ͷͰKVMΛ௨ͯ͡సૹ͞ΕΔ • ࠷ੑೳ͕ߴ͍͕ϋʔυରԠ͕ඞཁʗήετͷύέοτΛϑΟϧλʔɾվม͢Δࣄ͍͠ NIC Χʔωϧ Χʔωϧ
ήετ Ϣʔβ PF PFυϥΠό qemu kvm VF υϥΠό TCP/IP ελοΫ NIC2 ϑΥϫʔυ ׂΓࠐΈ ཧׂΓࠐΈ ׂΓࠐΈ ׂࠐΈ ϋϯυϥ VF %."ύεεϧʔ ϋΠύʔόΠβ 1211݄20Ր༵
macvtap <interface type='direct'> <mac address='52:54:00:6b:28:01'/> <source dev='eth1' mode='vepa'/> <model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </interface> 1211݄20Ր༵
SR-IOV <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10'
function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </hostdev> 1211݄20Ր༵
ൺֱରͱͯ͠ͷ ࣮ػੑೳଌఆ • ̎ͷLinuxػɾ10G NIC • KVMήετͰnetserver࣮ߦ • 128ϓϩηεͷnetperf •
TCP Request/ResponseϞʔυ 1byteͷύέοτΛϐϯϙϯ VMϗετ ςετػ 10G NIC 10G NIC netperf netserver netperf netserver 1211݄20Ր༵
VMΛ1-64·Ͱ૿Ճ vhost-netͱSR-IOVൺֱ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ 0 1375.00 2750.00 4125.00 5500.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM transaction/sec ཧίΞʹ͍ۙลΓͰ ੑೳ࠷ߴ 1211݄20Ր༵
ϨΠςϯγ 0 750.00 1500.00 2250.00 3000.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM ϨΠςϯγ ཧίΞʹ͍ۙลΓͰ ϨΠςϯγ࠷ద 1211݄20Ր༵
CPUෛՙ 0 25.00 50.00 75.00 100.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM ύʔηϯςʔδ ෛՙΛVMͷίΞʹ ͔͠ࢄͰ͖ͯͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 30% 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 30% 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 20% 1211݄20Ր༵
ൺֱ݁Ռ • ཧίΞʹ͍ۙVMͩͱੑೳ͕Ұ൪ྑ͍Α͏ʹ ݟ͑Δ • ཧίΞͷ10ഒͷVMΛ࣮ߦͯ͠ɺ͜ͷෛՙύλ ʔϯͰۃͳϨΠςϯγͷѱԽෛՙͷภΓͳͲ ݟΒΕͳ͔ͬͨ • ཧڥͱൺ͔ͯͳΓੑೳ͕ѱ͍
→ෛՙΛෳίΞࢄग़དྷ͍ͯͳ͍ࣄ͕ݪҼͷީ ิͱͯ͠ߟ͑ΒΕΔ 1211݄20Ր༵
Ծڥ্Ͱ ෳίΞωοτϫʔΫIO ෛՙࢄͤͨ࣌͞ͷੑೳ 1211݄20Ր༵
Ծڥ্ͰίΞࢄʁ • ԾCPUΛ1-16·Ͱ૿ͯ͠ΈΔ • ίΞΛ૿͚ͨͩ͠ͰωοτϫʔΫε λοΫ͕ฒྻͰΒͳ͍ • ✕ RSSɿԾNICɾVF͕ະରԠ •
◦ RPSɿήετଆLinuxͷઃఆͰ༗ޮԽ͠ ͯΈΔ 1211݄20Ր༵
RPS $ echo "f" > /sys/class/net/eth1/queues/rx-0/ rps_cpus $ echo 4096
> /sys/class/net/eth1/queues/ rx-0/rps_flow_cnt $ echo 32768 > /proc/sys/net/core/ rps_sock_flow_entries 1211݄20Ր༵
vCPU <vcpu placement='static'>16</vcpu> 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢvhost-netʣ 0 1375.00 2750.00 4125.00 5500.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢvhost-net, RPSʣ 0 1375.00 2750.00 4125.00 5500.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒRPSͰ ੑೳվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢSR-IOVʣ 0 1375.00 2750.00 4125.00 5500.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢSR-IOV, RPSʣ 0 1375.00 2750.00 4125.00 5500.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒRPSͰ ੑೳվળ 1211݄20Ր༵
ϨΠςϯγ ʢvhost-netʣ 0 750.00 1500.00 2250.00 3000.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢvhost-net, RPSʣ 0 750.00 1500.00 2250.00 3000.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢSR-IOVʣ 0 750.00 1500.00 2250.00 3000.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢSR-IOV, RPSʣ 0 750.00 1500.00 2250.00 3000.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
CPUෛՙ ʢvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VM =< 4ͳΒɺcpu = 1 ͷ࣌ΑΓෛՙΛࢄͰ ͖͍ͯΔʁ 1211݄20Ր༵
CPUෛՙ ʢvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ RPSͰΑΓෛՙΛࢄ 1211݄20Ր༵
CPUෛՙ ʢSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
CPUෛՙ ʢSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ cpu >= 8, VM >=16ͷ࣌ʹ 15%Λ͑ΔภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯ/vhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ cpu >= 8, VM >=16ͷ࣌ʹ 15%Λ͑ΔภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ͋·Γେ͖ͳภ ΓݟΒΕͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ͋·Γେ͖ͳภ ΓݟΒΕͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VMͷ্ঢͰ एׯ૿Ճ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VMͷ্ঢͰ एׯ૿Ճ 1211݄20Ր༵
ൺֱ݁Ռ • ԾϚγϯͰ࣮ػੑೳͷҎ্Λग़ͤͨͷVM͕͘͝গͳ͘ vCPU͕͔ͳΓଟ͍࣌ͷΈ → ͜ͷ݅Ͱ͔͠RPSͰੑೳ͕ग़ͳ͍ • ૯vCPUΛଟͯ͘͠ҙ֎ͱੑೳԼ͕Βͳ͍ • SR-IOVͷ߹ʹVF͕ཧίΞΛ͑ͨลΓͰ͔ͳΓੑೳʹภ
Γ͕ͰΔ • NIC্ͷΩϡʔʹରͯ͠ड৴ॲཧʹׂΓͯΔίΞ͕ෆͯ͠ ͜ͷΑ͏ͳঢ়گʹͳΔͱਪଌ • vhost-netͰͦͷΑ͏ͳݱ͕ݟΒΕͣɺ૯vCPUΛ૿ͯ͠ ൺֱతެฏʹॲཧ͞Ε͍ͯΔΑ͏ʹݟ͑Δ 1211݄20Ր༵
ϚϧνϢʔβ࣌ͷެ ฏੑ 1211݄20Ր༵
ϚϧνϢʔβ࣌ͷެฏੑ • σϑΥϧτεέδϡʔϦϯάͰɺશͯͷKVM ήετϓϩηε͕ެฏʹCPU࣌ؒΛׂΓͯΒ ΕΔ →ෳͷϢʔβ͕ෳͷVMΛىಈ͍ͯ͠Δ ߹ɺϢʔβลΓͷϦιʔεׂ͕ෆެฏʹͳΔ • ϢʔβؒͰެฏʹεέδϡʔϦϯάͯ͠ཉ͍͠ 1211݄20Ր༵
cgroup • ϓϩηεάϧʔϓʹରͯ͠Ϧιʔε ʢCPUɺϝϞϦɺσΟεΫI/Oɺωοτ ϫʔΫI/Oʣͷར༻Λ੍ݶɾִ͢Δҝ ͷLinuxͷػೳ 1211݄20Ր༵
cgroup tool install $ apt-get install cgroup-bin 1211݄20Ր༵
$ ls /sys/fs/cgroup blkioɹϒϩοΫI/Oͷ੍ݶ cpuɹCPUϦιʔεͷ੍ݶ cpuacctɹCPUͷߦ࣌ؒͳͲͷใऩू cpusetɹCPUͷׂΓ͚ʹؔ͢Δ੍ݶ devicesɹσόΠεͷΞΫηεڐՄɾڋ൱ freezerɹάϧʔϓʹଐ͢ΔϓϩηεͷҰ࣌ఀࢭɾ࠶։ memoryɹϝϞϦϦιʔεͷ੍ݶ
net_clsɹૹग़ύέοτʹtc༻ͷλάΛઃఆ nsɹωʔϜεϖʔεʢίϯςφ༻ʣ 1211݄20Ր༵
cd /sys/fs/cgroup/cpu $ mkdir grp_a $ echo ‘12254’ > grp_a/tasks
$ echo ‘2184’ > grp_a/tasks $ echo ‘512’ > grp_a/cpu.shares $ mkdir grp_b $ echo ‘9012’ > grp_b/tasks $ echo ‘1024’ > grp_b/cpu.shares άϧʔϓ࡞ άϧʔϓॴଐϓϩη εΛՃʢPIDʣ άϧʔϓͷλεΫͰ༻ग़དྷ ΔCPU࣌ؒͷ૬ରత 1211݄20Ր༵
libvirtdͱcgroup • /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>ʹ VMؔ࿈ϓϩηε͕ࣗಈొ͞ΕΔ • /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/ vcpuNʹvcpuϓϩηε͕ࣗಈొ͞ΕΔ • libvirtܦ༝ͰcgroupͷϦιʔε੍ݶઃఆ ͕ग़དྷΔ
1211݄20Ր༵
virshͱcgroup $ virsh schedinfo vm0 Scheduler : posix cpu_shares :
1024 vcpu_period : 100000 vcpu_quota : -1 $ virsh schedinfo --set cpu_shares=512 vm0 1211݄20Ր༵
̎ͭͷάϧʔϓʹެฏʹϦ ιʔεΛׂΓͯͯΈΔ • /sys/fs/cgroup/grp_a • VMΠϯελϯεɿ̒ݸ • cpu.sharesɿσϑΥϧτʢ̍̌̎̐ʣ • /sys/fs/cgroup/grp_b
• VMΠϯελϯεɿ̍̌ݸ • cpu.sharesɿσϑΥϧτʢ̍̌̎̐ʣ 1211݄20Ր༵
άϧʔϓͷ࡞Γํ $ virsh start <VM໊> $ mkdir /sys/fs/cgroup/cpu/grp_a $ cat
/sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/tasks > tmp $ cat /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/ vcpu*/tasks >> tmp $ cat tmp > /sys/fs/cgroup/cpu/grp_a/tasks 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢcgroupະ༻࣌ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢcgroup༻࣌ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ grp_a grp_b
1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢcgroupະ༻࣌ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ 1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢcgroup༻࣌ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ grp_a grp_b
1211݄20Ր༵
άϧʔϓ͝ͱͷ߹ܭൺֱ ʢcgroup༻࣌ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.10 1.14 1211݄20Ր༵
ʦࢀߟʧάϧʔϓ͝ͱͷ߹ܭൺֱ ʢcgroupະ༻࣌ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.64 1.65 1211݄20Ր༵
ൺֱ݁Ռ • άϧʔϓʹ͚Δ͚ͩͰσϑΥϧτͰάϧʔϓεέ δϡʔϦϯά͕ߦΘΕɺ࣮ଌͰάϧʔϓʹର͠ ͯެฏʹCPU͕͞Ε͍ͯΔࣄ͕֬ೝͰ͖Δ • CPUΛߜΔࣄͰ݁ՌతʹωοτϫʔΫIOΛߜΔࣄ ͋Δఔग़དྷ͍ͯΔ • Ͱ͋·Γݫີ͡Όͳ͍ʢ࣮ଌ14%͘Β͍ͷޡࠩʣ
1211݄20Ր༵
CFS bandwidth control • ͜ͷάϧʔϓcpu.cfs_period_usͷ୯Ґ࣌ ؒதʹ࠷େcpu.cfs_quota_us͚࣮ͩޮग़དྷΔ • άϧʔϓͷϓϩηεperiod୯Ґ࣌ؒ தɺಉ࣌ʹ࣮ߦ͞Ε͏ΔͷͰquotaperiod Λ͑͏Δʢ2CPUͰΊ͍͍ͬͺ͍ͬͨ
Βquota = period * 2ʣ 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢquota=200000,period=50000ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ grp_a grp_b
1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢquota=200000,period=50000ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ grp_a grp_b
1211݄20Ր༵
άϧʔϓ͝ͱͷ߹ܭൺֱ ʢquota=200000,period=50000ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.01 0.63 1211݄20Ր༵
ݕূ݁Ռ • ͔֬ʹάϥϑ্៉ྷʹͳΔͷ͕ͩɺCPU͕ͲΜͳʹ ۭ͍͍ͯͯquota·ͰߦͬͨΒઈର࣮ߦ͞Εͳ͍ • ͔ͱ͍࣮ͬͯߦ͖͠Εͳ͍ఔͷquotaΛ༩͑Δͱઃఆ͠ ͳ͍߹ͱมΘΒͳ͘ͳΔ • CPUωοτϫʔΫIOͪΐ͏ͲΑ͘ͽͬͨΓ1.0ͱ͍ ͏༁ʹ͍͔ͳ͔ͬͨ
• period͕͗͢ΔͱVM͕͍࣮ؒߦ͞Εͳͯ͘ྑ͘ͳ ͍͜ͱ͕ى͖ͦ͏ 1211݄20Ր༵
net_cls $ tc class add dev virbr0 parent 10: classid
10:1 htb rate 24Mbit $ echo 0x100001 > /sys/fs/cgroup/net_cls/ grp_a/net_cls.classid ϓϩηε͔Βૹ৴͞ΕΔύέοτʹtcͷ λάΛ͚ΔࣄʹΑΓଳҬ੍ݶ͕ग़དྷΔ 1211݄20Ր༵
·ͱΊ • KVMʹΑΔԾԽڥͷϦιʔεׂΓͯͷެฏੑʹ͍ͭͯ ݕূͨ͠ • ࠓճͷ࣮ݧͷ݅ԼͰɺ࣮CPUΛେ͖͑͘ΔVMͰ ۃͳϨΠςϯγͷѱԽෛՙͷภΓͳͲݟΒΕͳ͔ ͬͨ • ୠ͠ɺSR-IOVͷ߹ʹVF͕ཧίΞΛ͑ͨลΓͰ͔
ͳΓੑೳʹภΓ͕ൃੜͨ͠ • cgroupΛ༻͍ͯϚϧνϢʔβ࣌ʹϢʔβؒͷCPUׂΓͯΛ ެฏʹग़དྷΔࣄΛ֬ೝͨ͠ 1211݄20Ր༵