Share
Embed
CRIUとSeccomp / criu-and-seccomp-and-me
第14回 コンテナ技術の情報交換会@オンライン
https://ct-study.connpass.com/event/205571
KONDO Uchio
April 17, 2021
Transcript
61 slides
-
ۙ౻Ӊஐ࿕ / GMO Pepabo, Inc. ୈ14ճ ίϯςφٕज़ͷใަձ@ΦϯϥΠϯ 2021/04/17 CRIUͱseccompͱ ֨ಆͨ͠
ʙશͯരىಈͷͨΊͩͬͨʙ *NBHFIUUQTQJYBCBZDPNJNBHFTJECZ"OUSBOJBT -
γχΞɾϓϦϯγύϧΤϯδχΞ ۙ౻ Ӊஐ࿕ / @udzura https://blog.udzura.jp/ Uchio Kondo ٕज़෦ ٕज़ج൫νʔϜ
@ GMOϖύϘ ΤϯδχΞΧϑΣʢԬࢢẂנจԽձؗʣ αϙʔλʔ #Ruby #mruby #Rust #Containers #eBPF #CRIU #Seccomp #RubyKaigi #CloudNativeDays #Zumba #γϨϯ #FitBoxing2 --- ͖ͳγεςϜίʔϧʁ ͬͺΓ unshare(2) Ͱ͢Ͷɻ -
ToC •࡞CRIUϥούʔ “Miehistö” ʹ͍ͭͯ •ҙͷΞϓϦέʔγϣϯΛҙͷՕॴͰμϯϓ͢Δٕज़ͷ࣮ݱ •seccomp + SCMP_ACT_TRACE ʹΑΔख๏ •seccomp
+ SCMP_ACT_NOTIFY ʹΑΔख๏ wͦͯ͠આ -
࠷ۙͷCRIU •3.13(Sep 11, 2019) ... libcriu.a ͕ϏϧυͰ͖ΔΑ͏ʹͳͬͨ by @udzura •3.14(π,
Apr 29, 2020) ... clone3(2)ͱTime NS support, ଞ •3.15(Nov 04, 2020) ... MIPS support, cgroup v2 support, PID NS෦ ͷϦετΞ, ... ଞ •Still developing... ຊൃදͷ$3*6ͷόʔδϣϯͰ͢ -
Miehistö (Έ͑ͻͯ͢) • (ex. Grenadine) • Miehistö = “CREW” in
Finnish NJF 㷺IJTU -
Miehistö ͱ •CRIUΛͳΔ͘ී௨ͷϓϩηεʹରͯ͠ద༻͘͢͢͠ΔɺҰ࿈ͷ πʔϧϥούʔ •miehistod: αʔϏεΠϝʔδΛ ཧ͢ΔதԝσʔϞϯ •mhctl: ΫϥΠΞϯτ •runmh:
CRIU͍͢͠ϓϩηεΛ ࡞ΔҰछͷϥϯλΠϜ Έ͑ͻͯ͢Ͱ͌ʔ Ήʔ͜ΜͱΖʔΔ ΒΜΉʔ -
۩ମతʹ... •·ͣɺPID Namespace Λ͠ɺPIDΛ1͔Β࢝ΊΔඞཁ͕͋Δ •→ clone(2) ʹΑΓִ͠ɺ /proc ϑΝΠϧγεςϜΛࣗͰϚϯτ •→
ͦͷͨΊɺMount namespaceִ -
Mount namespace/root ͷ •/proc ΛϚϯτ͢͠ͷͰMount NSunshared •ಠཱͨ͠ɺ͔ͭϗετͱେମಉ͡root filesystem͕ඞཁ •࡞Γํ: •ԾͷσΟϨΫτϦʹ
/ Λbind mountɺ/devͳͲݸผʹbind mount •ͦ͜ʹpivot_root͢Δ(chroot μϝɺͦͷMount NSͰೝࣝ͞ΕΔ rootΛૠ͛ସ͑Δඞཁ͕͋ΔͨΊ) -
࣮ͷΠϝʔδ /&8/4 /&81*%ͷϑϥάΛ༩͑ͯDMPOF )PTUSPPU ΛผͷՕॴʹCJOENPVOU TFUTJE QJWPU@SPPU GEΛEFWOVMM
GE ΛݱࡏͷSPPU෦ͷϑΝΠϧʹ͠ 0@830/-:c0@"11&/%Ͱ։͘ ରϓϩάϥϜʹFYFD -
ϦετΞʹ͍ͭͯ •Miehistöͷཁ্݅ɺϦετΞ͞Εͨϓϩηεmiehistod(runmh)Ͱ ཧ͍ͨ͠ɻͳͷͰҙͷϓϩηεͷࢠϓϩηεͱͯ͠ϦετΞ͢Δඞ ཁ͕͋Δ... ͦΜͳͷͰ͖Δʁ •ӈͷΑ͏ͳ͜ͱΛ͢Δࡍɺ runmh -> criu ->
ruby criu͕ফ͑ͨΒ... μϝ ࢦ͍ͯ͠ΔϓϩηεπϦʔ -
miehistö ͰͷϦετΞ࣮ •miehistod ͷԼͰ criu restore ΛݺͿ •ͦͷࡍʹɺ --exec-cmd ͱ͍͏ΦϓγϣϯΛར༻͠ɺ
ϦετΞޙʹcriuίϚϯυࣗΛrunmhϓϩάϥϜʹexec͢Δ • ΛrunmhɺࢠΛϦετΞޙͷίϚϯυͱ͍͏ϓϩηεπϦʔΛ࡞͍ͨͨ͠Ί •·ͨɺbind mountͷѻ͍ʹ͍ͭͯcriu restoreʹ͢ඞཁ͕͋Δ •--external Φϓγϣϯʹ͍ͭͯ -
External bind mounts •root͔Βݟͯ֎෦σΟϨΫτϦΛBind mount͍ͯ͠Δใɺdump/ restore࣌ʹࣗಈݕ͞Εͳ͍ͷͰɺ໌ࣔతʹࢦఆ͢Δඞཁ͕͋Δɻ •dump࣌ʹˠ ֎෦Bind mountઌΛنͰܾΊ͍ͯΔͷͰɺͦΕΒͷ ใΛ
--external ૬ͷΦϓγϣϯͱͯ͠criuαʔϏεʹ͢ •restore࣌ʹˠ ҎԼͷΑ͏ͳܗࣜͰίϚϯυʹ͢ --external mnt[__pids-d05739ddd0a12dd5040494c20461a197]:/sys/fs/cgroup/pids IUUQTDSJVPSH&YUFSOBM@CJOE@NPVOUT -
cf. strace -c rails s •rails newͯ͠΄΅CRUDͻͱͭՃ͚ͨͩ͠ͷΞϓϦɺͷىಈ RAILS_ENV=production •΄΅openͱstat ϑΝΠϧૢ࡞
•͜ΕΒͷopenΛશ෦ εΩοϓͰ͖Εͦ͏ -
SCMP_ACT_TRACE ͍ํ • fork͢Δ • [] ptrace(PTRACE_ATTACH); ptrace(PTRACE_O_TRACESECCOMP) ͢Δ •
[] ptrace(PTRACE_CONT) ͢Δ • [] waitpid(-1, &status, WUNTRACED | WCONTINUED | __WALL) ͢Δ (-1 ͳͷtraceͯ͠Δ͞ΒʹࢠϓϩηεɾεϨουΛ͏ͨΊ) • [ࢠ] SCMP_ACT_TRACE ͳseccomp ctxΛϩʔυ͢Δ • [] ࢠͰ֘γεςϜίʔϧ͕ݺΕΔͱɺ֘ϓϩηεͷใ͕waitpidͷΓɺ ptrace(PTRACE_GETEVENTMSG)ɺptrace(PTRACE_GETREGS)ͳͲͰऔಘͰ͖Δ -
ptrace ͷΦϓγϣϯ͕όοςΟϯά͢Δʁ •seccompͰͷτϨʔεϓϩηεΛ ptrace(PTRACE_ATTACH) ͯ͠Ξ λονঢ়ଶʹ͠ɺఀࢭঢ়ଶΛݕ͢Δ •Ұํɺcriuͷ෦Ͱɺ ptrace(PTRACE_SEIZE) ͰΞλονঢ়ଶʹͯ͠ ptrace(PTRACE_INTERRUPT)
Ͱ໌ࣔతʹࢭΊ͍ͯΔɻ •ผʑͷΦϓγϣϯͰࢭΊ͍ͯΔͷʹͳ͍ͬͯΔՄೳੑ͕͋Δ •͜ͷลΓͷύονΛແཧʹcriuʹͯͯ... • ݁ہseccomp ctx͕ϓϩηε͔Βൈ͚ͳ͍ͨΊɺ2ճͷ listen() ͕ptrace tracee͕ଘࡏ͠ͳ͍ѻ͍ʹͳΓENOSYS ʹͳΔ? -
࠷ޙͷ࣮ݧ •ϥούʔΛ͞Βʹࠨʹมߋ •ىಈˠ Notification receiver ܦ༝Ͱμϯϓ LJMM BOZ ͷݺͼग़͠Λτϥοϓ
4*(35."9 -%@13&-0"%ΛFYFD࣌ʹࢦఆ -
ࢀߟهࣄͳͲ •அଓతʹॻ͍ͯΔseccompͱMiehistöͷϒϩά • ʮmrubyͱseccompͱptraceͰγεςϜίʔϧΛͱʹ͔͍͔͚͘Δʯ(2017/04) • ʮGrenadine: ʮී௨ͷΞϓϦέʔγϣϯʯ͕νΣοΫϙΠϯτ/ϦετΞͷԸܙΛڗड͢Δʯ (2019/03) • ʮWSAݚڀձ
ୈ7ճ ͰCRIUͱMiehistöͷൃදΛ͠·ͨ͠ #WSAݚʯ(2020/11) • ʮҙͷϥΠϒϥϦίʔϧͰϓϩάϥϜΛఀࢭ͠ɺىಈ༻ͷCRIUΠϝʔδΛ࡞͢ΔΞϓ ϩʔνʹ͍ͭͯʯ(2020/12) IUUQTVE[VSBIBUFOBCMPHKQ