Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
3 state-of-the-art technologies in Linux and fu...
Search
KONDO Uchio
February 21, 2021
Technology
2
630
3 state-of-the-art technologies in Linux and future of the containers #SECKUN
2021.02.21 「新しいセキュリティビジネスキャリア」シンポジウム
KONDO Uchio
February 21, 2021
Tweet
Share
More Decks by KONDO Uchio
See All by KONDO Uchio
大規模レガシーテストを 倒すための CI基盤の作り方 / #CICD2023
udzura
5
2.4k
Ruby x BPF in Action / RubyKaigi 2022
udzura
0
220
Narrative of Ruby & Rust
udzura
0
200
開発者生産性指標の可視化 / pepabo-four-keys
udzura
3
1.7k
Talk of RBS
udzura
0
420
Re: みなさん最近どうですか? / FGN tech meetup in 2021
udzura
0
750
Dockerとやわらかい仮想化 - ProSec-IT/SECKUN 2021 edition -
udzura
2
710
Device access filtering in cgroup v2
udzura
1
840
"Story of Rucy" on RubyKaigi takeout 2021
udzura
0
780
Other Decks in Technology
See All in Technology
Multitenant 23ai の全貌 - 機能・設計・実装・運用からマイクロサービスまで
oracle4engineer
PRO
2
120
OPENLOGI Company Profile for engineer
hr01
1
22k
Compose MultiplatformにおけるiOSネイティブ実装のベストプラクティス
enomotok
1
210
グループポリシー再確認
murachiakira
0
170
モノリスの認知負荷に立ち向かう、コードの所有者という思想と現実
kzkmaeda
0
110
OPENLOGI Company Profile
hr01
0
61k
お問い合わせ対応の改善取り組みとその進め方
masartz
1
370
ClineにNext.jsのプロジェクト改善をお願いしてみた / 20250321_reacttokyo_LT
optim
1
1.3k
Security response for open source ecosystems
frasertweedale
0
100
年末調整プロダクトの内部品質改善活動について
kaomi_wombat
0
210
非エンジニアにも伝えるメールセキュリティ / Email security for non-engineers
ykanoh
13
3.9k
移行できそうでやりきれなかった 10年超えのシステムを葬るための戦略 / phper-kaigi-2025-ryu
carta_engineering
0
690
Featured
See All Featured
Build your cross-platform service in a week with App Engine
jlugia
229
18k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Rails Girls Zürich Keynote
gr2m
94
13k
Product Roadmaps are Hard
iamctodd
PRO
52
11k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
4
470
Speed Design
sergeychernyshev
28
860
The Invisible Side of Design
smashingmag
299
50k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.1k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Designing for humans not robots
tammielis
250
25k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Transcript
ཁૉٕज़ͷstate of the art͔Βߟ͑Δ ۙ౻Ӊஐ࿕ / GMO Pepabo, Inc. 2021.02.21
ʮ৽͍͠ηΩϡϦςΟϏδωεΩϟϦΞʯγϯϙδϜ Linuxίϯςφͷະདྷ
GMOϖύϘגࣜձࣾ γχΞϓϦϯγύϧ ٕज़෦ٕज़ج൫νʔϜॴଐ ۙ౻Ӊஐ࿕ (@udzura)
ۙ౻Ӊஐ࿕ ུྺ • ࡾՏᅳͷਓɻچ٢ాൡߍ࣌शؗߴߍΛଔۀɺ౦ژେֶจֶ෦ຊޠຊจֶઐम՝ఔ ͷֶ࢜ଔʢ2007ʣɻ • ϚείϛͷࣾSEɺECαΠτ։ൃɺΦϯϥΠϯήʔϜ ։ൃͳͲΛܦͯ2013ΑΓݱ৬ɺಉʹԬҠॅɻ • RubyɺίϯςφɺΫϥυωΠςΟϒٕज़ͳͲͷίϛϡχςΟͰ
׆ಈɻஶॻʹʮWebͰ͑ΔmrubyγεςϜϓϩάϥϛϯάೖʯʢC&Rݚڀॴʣ • ͖ͳγεςϜίʔϧʢ࠷ۙʣ socketpair(2) ɻ
ࠓͷ͓ •ίϯςφͷཁૉٕज़ʹ͍ͭͯɺࢲͷߨٛͰʢͬ͘͟Γʣཧղ͞Εͨํ ͚ͷ༰Ͱ͢ɻཁૉٕज़ͷղઆࢀߟࢿྉΛͲ͏ͧɻ •ࢀߟ1: https://container-security.dev/ •ࢀߟ2: ʰίϯςφܕԾԽ֓ʱʢޱ, ΧοτγεςϜʣ •ۙͷΧʔωϧʹؚ·ΕΔ৽ٕज़ͷ͏ͪɺίϯςφʹؔ͢ΔͷΛ հ͠·͢ɻ
•͕࣌ؒ͋Εɺͦͷ্ͰίϯςφͷະདྷΛߟ͑·͢ɻ
cgroup v2
cgroup ͷ͓͞Β͍ •Linux Kernelͷجຊٕज़ͷҰͭɻ •ϓϩηεΛάϧʔϐϯά͠ɺͦͷάϧʔϓ୯ҐͰϦιʔεར༻ͷ੍ݶ Λ͔͚Δٕज़ɻCPUɺϝϞϦɺIOɺϓϩηε... IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFST
cgroup v1ͷྫ •cgroupfs ͱ͍͏ϑΝΠϧγεςϜʹmkdir(2) read(2) write(2)ͳͲΛ࣮ ߦ͠ɺૢ࡞Λߦ͏
cgroup v2 •v1 ͷ͍͔ͭ͘ͷܽ - ओʹ੍ޚରʢίϯτϩʔϥʣ͝ͱʹσΟϨΫ τϦΛ͚ͳ͚Ε͍͚ͳ͍༷ - Λࠀ͘͢։ൃ͞Εͨ •େ͖ͳҧ͍ͱͯ͠ɺ
v1 ͰίϯτϩʔϥผʹσΟϨΫτϦ ΛϚϯ τɺݸผʹάϧʔϓʹॴଐͰ͖ͨͷʹର͠ɺv2ͰશίϯτϩʔϥΛ ·ͱΊͨҰͭͷσΟϨΫτϦͷΈΛϚϯτ͠ɺ·ͱΊͯάϧʔϓΛ ࡞͢ΔڍಈʹͳΔɻ •ίϯςφͱͯͪ͜͠Βͷํ͕߹͕͍͍ɻ
Unified hierarchy /sys/fs/cgroup /sys/fs/cgroup /group-a /group-b /cpu.* /memory.* /io.* ...
/cpu.* /memory.* /io.* ... /cpu /memory /blkio /group-a /group-b /group-a /group-c
ίϯςφϥϯλΠϜͰͷcgroupͷར༻ •ʢOCIܥͷʣϥϯλΠϜͰҎԼͷ2ͭͷઃఆ߲͕͋Δ •Cgroup Driver: ίϯςφʹׂΓͯΔcgroupΛͲ͏ίϯτϩʔϧ͢Δ͔ •cgroupfs: cgroupfsͷͷϑΝΠϧૢ࡞ •systemd: systemdʹΑΔཧ •Cgroup
Version: Ϧιʔε੍ݶʹ v1/v2 ͲͪΒΛར༻͢Δ͔ •/sys/fs/cgroup ʹͲͷϑΝΠϧγεςϜ͕Ϛϯτ͞ΕͯΔ͔Ͱఆ
v2 ͷ৽ػೳ •Unified Hierarchy •PSI(Pressure Stall Information) •eBPFͰcgoup IDͷऔಘ͕Մೳʹ •nsdelegate
(ඇಛݖίϯςφʹॏཁ) •clone3(2) Ͱಛఆͷcgroup෦ʹϓϩηε࡞͕Մೳʹ •ͳͲͳͲ...
e.g. PSI(Pressure Stall Information) •γεςϜશମɺ·ͨcgroup୯ҐͰར༻Ͱ͖Δෛՙͷࢦඪ •CPU, ϝϞϦ, IO Ͱ stall
ͨ͠୯Ґ࣌ؒͰͷׂ߹ ΛܭଌͰ͖Δ •e.g. 1ؒͰ45ඵؒɺάϧʔϓͷ ͋Δϓϩηε͕CPUىҼͰ Ԇͨ͠߹ɺcpu some: 75.00
e.g. eBPFͰͷτϥοΩϯάใ •bpf_get_current_cgroup_id(void) ϔϧύʔ •eBPFͷΠϕϯτ͕ى͖ͨλεΫ͕Ͳͷcgroup(v2)ʹॴଐ͍ͯ͠Δ͔ɺ ͦͷIDΛฦ͢ɻ
How cgroup-v2 and PSI Impacts Cloud Native? Uchio Kondo /
GMO Pepabo, Inc. 2019.07.23 CloudNative Days Tokyo 2019 Image from pixabay: https://pixabay.com/images/id-3193865/
eBPF per containers
eBPFٕज़ͱ •ϢʔβۭؒͰ࡞ͬͨϓϩάϥϜΛΧʔωϧͰಈ͔ٕ͢ज़ͷͻͱͭ •2012ʹseccompͷಋೖɺ2013ʹLinuxͷSDNͰͷԠ༻͕࣮͞ ΕɺͦΕҎ߱ख़͢Δ •ϑΟϧλϦϯά͕ಘҙʢtcpdump, seccomp, bpftraceʣ •ΧʔωϧͷใʹΞΫηεͰ͖Δ͕ɺةݥͳίʔυಈ͔ͳ͍ͳͲ ҆શੑ͕͋Δఔ୲อ͞Ε͍ͯΔ
eBPFͷԠ༻ྫ
ίϯςφͷeBPFτϨʔεઓུ •ઓུ͕͍͔ͭ͋͘Δ •Linux Namespace·ͨcgroup (v2)ͷใ͕ར༻Ͱ͖Δ
ྫ1: task_struct ͷใΛḷΔ •task_struct→nsproxy ͔Β namespaceͷใΛ औಘͯ͠ϑΟϧλ͢Δ ʢcxrayʣ IUUQTHJUIVCDPNNSUDDYSBZCMPCNBTUFSQLHUSBDFSPQFOPQFOHP--
ྫ2: NS/ϗετͰͷPIDΛൺֱ •BPFϓϩάϥϜͰऔಘͰ͖ͨ tidͱɺϗετͰͷtidΛ ൺֱ͠ɺҰக͠ͳ͚Ε ίϯςφͱఆ͢Δ ʢTraceeʣ • task_structґଘ IUUQTHJUIVCDPNBRVBTFDVSJUZUSBDFFCMPCNBJOUSBDFFUSBDFFCQGD-ɹ
ྫ3: cgroup helperΛར༻ IUUQTHJUIVCDPNVE[VSBDPQFODMPTFCMPCNBTUFSTSDCQGDPQFODMPTFCQGD
࣮ྫ •copenclose(8) •ۙ౻ͷPoC (BPF+Rust) •ϑϥάͰtask_struct/ cgroup v2 ID ΛΓସ͑
bpf_get_current_cgroup_id(void) を添えて Uchio Kondo / Container Runtime Meetup #3 ランタイムとcgroupの
xxxな関係 * Photo by Fukuoka City
seccomp
seccompͷ͓͞Β͍ •ϓϩάϥϜʹ͓͚ΔγεςϜίʔϧݺͼग़͠ΛϑΟϧλϦϯά͢Δ •γεςϜίʔϧͷҾͷ݅ʹΑͬͯࢦఆΛม͑ΒΕΔ •blacklist(denylist), whitelist(allowlist) ͳͲΛ࣮Ͱ͖Δ •ϑϥά͕ࡉ͔͘ଘࡏ͠ɺྫ͑γεςϜίʔϧͷauditϩάͷΈɺҙ ͷerrnoΛฦͤ͞ΔɺͳͲͷࢦఆ͕Ͱ͖Δ
seccompͷར༻(mruby)
User space notification •seccompʹΑΓγεςϜίʔϧݺͼग़͠Λݕ͠ɺͦͷڍಈΛϢʔβ ϥϯυͷϓϩάϥϜʹҕͶΔ͜ͱ͕Ͱ͖Δٕज़ •Linux 5.0 (2019/3) ͔Βͷಋೖ •அ͢Δ·ͰɺͦͷγεςϜίʔϧϒϩοΫ͢Δ
•e.g. LXCͰͷσόΠεΞΫηεͷ੍ޚ IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFSTɹ
User space notification IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFSTɹ • LXCͰֶͿίϯςφೖ ୈ47ճɹඇಛݖίϯςφͷՄೳੑΛ͛Δseccomp notifyػೳ ΑΓ
࣮ྫʢmrubyར༻ʣ •ҎԼͷΑ͏ͳ acceptor.rbΛ ༻ҙ͢Δ
࣮ྫʢmrubyར༻ʣ •ҎԼͷinvokerΛܦ༝ͯ͠ϓϩάϥϜΛ ىಈɺ listen(3) ΛݺͿ
listen(2) ͷىಈݕ •acceptor.rb ଆͷίϯιʔϧͰڐՄ/ېࢭΛ੍ޚՄೳɻ •ېࢭͨ͠Βͦͷ··ىಈࣦഊͯ͠invokerϓϩηε͕མͪΔ •ڐՄͨ͠ΒԿͳ͔͔ͬͨͷΑ͏ʹɺىಈΛܧଓͯ͠Ϧοεϯɻ
listen(2) ͷىಈݕ ېࢭ࣌ͷग़ྗ ڐՄ࣌ͷग़ྗ
Ԡ༻ʁ •ʮҙͷϥΠϒϥϦؔݺͼग़͠ʯͰϓϩηεΛఀࢭɺCRIU(*)ʹΑΓ ϓϩηεμϯϓΛ࡞͢Δ࣮ݧΛߦͬͨɻ •LD_PRELOAD + ϥούؔ + ʮԿ͠ͳ͍ʯsyscall + seccomp
IUUQTVE[VSBIBUFOBCMPHKQFOUSZ $IFDLQPJOUBOE3FTUPSF*O6TFSTQBDF ϓϩηεͷঢ়ଶΛอଘɺ͔ͦ͜Β࠶ੜ͢Δٕज़ IUUQTDSJVPSH.BJO@1BHF
ߟ
৽ٕज़ʹΑΓͰ͖Δ͜ͱ૿͑Δ͕... •৽ٕज़ͷʮग़ݱʯͱʮීٴʯͷλΠϛϯάζϨΔ •ͨͱ͑ cgroup v2ͷॳग़2013ɻ •2019 ~ 2020 ʹϥϯλΠϜͰͷରԠ͕ਐΜͩଆ໘ •पลͷπʔϧ͕ग़ݱ͢ΔͷͬͱઌͰ͋Ζ͏
•ग़ݱظʹ୯ମͰٕज़Λݕূ͠ɺʢηΩϡϦςΟؚΊʣͲ͏͍͏͕ ͋Δ͔ɺͲ͏͍͏Մೳੑ͕͋Δ͔ݕূ͢Δҙٛେ͖͍
eBPF Linuxͷجຊٕज़ʹͳΓͭͭ͋Δ •ద༻ൣғ͕ͲΜͲΜ·͍ͬͯΔ •τϨʔγϯάɺଳҬ੍ޚωοτϫʔΩϯάͷ΄͔ɺcgroup(v2) deviceͷఆɺLSM BPF programͳͲͳͲ... •ηΩϡϦςΟͷจ຺ͰτϨʔεɺࠪɺҟৗݕͱ͔ܽͤͳ͍ٕज़ ʹͳΔ͜ͱ͕૾͞ΕΔ •Ұͭͷprog
typeʹ৮͓͚ͬͯͩ͘Ͱײ͔֮Γͦ͏