Compare OCI Runtimes

Transcript

1. ͘͞ΒΠϯλʔωοτגࣜձࣾ (C) Copyright 1996-2019 SAKURA Internet Inc ͘͞ΒΠϯλʔωοτݚڀॴ OCIϥϯλΠϜൺֱͷͨΊʹ ΍͍ͬͯΔ͜ͱ͋Ε͜Ε 2019/03/22 ٬һݚڀһ ٶԼ ߶ี runc, gVisor, Kata Containers Nabla Containers, Firecrackerൺֱ
2. 1. ͸͡Ίʹ
3. 3 ɾݱࡏOCIϥϯλΠϜͷൺֱʹऔΓ૊ΜͰ͍Δ ɾൺֱ߲໨͸ɺىಈ࣌ؒɺϝϞϦαΠζɺύϑΥʔϚϯεͳͲ ɾϥϯλΠϜຖʹบ͕͋Γɺಉ͡৚݅Ͱൺֱ͢Δͷ͕؆୯Ͱ͸ͳ͍ ɾ۩ମతʹ΍ͬͨ͜ͱɺͱ͘ʹ͸·ͬͨ͜ͱڞ༗͍ͨ͠ ɾܭଌ݁Ռͷڞ༗͸͜ͷൃදͷ໨తͰ͸ͳ͍ ɾݱࡏͷܭଌ݁Ռʹ͍ͭͯ͸matsumotory͞ΜͷεϥΠυࢀর ɾhttps://speakerdeck.com/matsumoto_r/chao-ge-ti-xing- detasentaostoocirantaimu ͓࿩͢͠Δ಺༰
4. 1. ܭଌ४උͰ͸·ͬͨ͜ͱ
5. ίϯςφͰಈ͔͢όΠφϦ࡞੒
6. 6 ɾൺֱ৚݅Λἧ͑ΔͷͱɺϥϯλΠϜͦͷ΋ͷͷੑ࣭Λଌఆ͍ͨ͠ͷ Ͱɺ୯७ͳϓϩάϥϜΛ༻ҙ ɾhello.c ɾHelloͱදࣔ͢Δ͚ͩͷϓϩάϥϜ ɾىಈ࣌ؒଌఆ༻ ɾloop.c ɾແݶϧʔϓ͢ΔϓϩάϥϜ ɾϝϞϦαΠζଌఆ༻ ίϯςφͰಈ͔͢όΠφϦ
7. 7 hello.c #include <stdio.h> void main() { printf("Hello\n"); }
8. 8 loop.c #include <stdio.h> void main() { int i = 0; while(1) { printf("%d\n", i++); } }
9. 9 ɾNabla Containers͸ϥΠϒϥϦOSʢϢχΧʔωϧʣܕΞϓϩʔνͷ ϥϯλΠϜ ɾ࣮ߦόΠφϦͱΧʔωϧ͕ҰମԽ ɾͳͷͰNabla Containers༻όΠφϦ͸ผʹ༻ҙ͢Δඞཁ͕͋Δ ɾx86_64-rumprun-netbsd-gcc -o hello.out hello.c ɾrumprun-bake solo5_ukvm_seccomp hello.nabla hello.out ɾ৽͠໨ͷϦϏδϣϯͩͱsolo5_ukvm_seccompͰ͸ͳ͘spt Nabla Containers༻όΠφϦ
10. 10 ɾnabla-containers/solo5ΛίϯύΠϧͯ͠Ͱ͖ͨsolo5.oΛ/usr/lib/ libsolo5_seccomp.aʹίϐʔ ɾnabla-containers/runnc ͸ϦϏδϣϯb78fe29Λར༻ ɾnabla-containers/rumprun͸ϦϏδϣϯ8b01b3Λར༻ ɾߋʹҎԼͷύονΛ͋ͯΔ ɾhttps://github.com/rumpkernel/rumprun/issues/122 ɾhttps://github.com/rumpkernel/rumprun/pull/118 Nabla༻όΠφϦͷϏϧυ
11. 11 ɾrumprun-bakeίϚϯυʹ΋ύον ɾ࠷ޙʹ࣮ߦͯ͠ΔίϚϯυʹ-L/usr/libΛ௥Ճ ɾhttps://blog.cloudkernels.net/posts/build-a-nabla-docker-image/ ɾ͜ͷखॱʹ͕ͨͬͯ͠࠷৽ͷϦϏδϣϯͰϏϧυͯ͠΋͏·͘ಈ͔ ͳ͔ͬͨ ɾSolo5: ABORT: spt/net.c:36: Assertion `netfd >= 0' failed Nabla༻όΠφϦͷϏϧυʢ͖ͭͮʣ
12. Kata Containersͷόʔδϣϯ
13. 13 ɾhttps://github.com/kata-containers/documentation/blob/master/ install/ubuntu-installation-guide.md ɾܭଌ༻ϗετʹUbuntuΛར༻ͨ͠ͷͰ͜ͷखॱʹैͬͨ ɾhttp://download.opensuse.org/repositories/home:/ katacontainers:/releases:/${ARCH}:/master/xUbuntu_$ (lsb_release -rs)/ ͕aptϦϙδτϦͱͯ͠ઃఆ͞ΕΔ ɾ͕ɺ͜Εͩͱ1.6rc1͕Πϯετʔϧ͞Ε·ͱ΋ʹಈ͔ͳ͔ͬͨ ɾs/master/stable-1.5/ Ͱղܾ Kata ContainersͷΠϯετʔϧ
14. 1. ܭଌ࣌ʹ͸·ͬͨ͜ͱ
15. ϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌ
16. 16 ɾ༨෼ͳϨΠϠʔΛল͍ͯͳΔ΂͘ૉͷঢ়ଶͰͷܭଌ͕໨త ɾOCI Filesystem BundleΛ༻ҙ ɾdocker export `docker create mizzy/hello` | tar -C bundle/rootfs - xvf - Ͱrootfsੜ੒ ɾrunc specͰconfig.jsonੜ੒ ϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌ
17. 17 ɾrunc, gVisor, Kata Containers, Nabla Containersʹ͸OCI Filesystem BundleΛ࣮ߦ͢ΔίϚϯυ͕͋Δ ɾFirecrackerʹ͸(·ͩ?)ଘࡏ͠ͳ͍ͬΆ͍ ɾͳͷͰFirecracker͸௚઀࣮ߦํࣜͰ͸ܭଌͰ͖ͳ͔ͬͨ ɾkata-fc࢖͑͹Ͱ͖ͦ͏ʢະணखʣ ɾhttps://github.com/kata-containers/documentation/wiki/Initial- release-of-Kata-Containers-with-Firecracker-support OCI Filesystem Bundle࣮ߦίϚϯυ
18. 18 ɾrunnc͸ଞͷίϚϯυͱҧ͍runαϒίϚϯυ͕ͳ͍ ɾcreateͯ͠start͢Δඞཁ͕͋Δ ɾtime runnc startͰܭଌ͠Α͏ͱ͢Δͱίϯςφ࣮ߦऴྃલʹtime ͷ݁Ռ͕ฦΔ → ྑ͍ܭଌํ๏໛ࡧத ɾconfig.jsonͷhooks.prestartͰωοτϫʔΫ·ΘΓͷઃఆΛߦ͏ඞ ཁ͕͋Δ ɾhttps://github.com/nabla-containers/runnc/issues/53 ɾconfig.jsonͰࢦఆ͢Δroot.path͕૬ରύεͩͱಈ͔ͳ͍ Nabla Containersׂ͕ͱۂऀ
19. containerdͷctrίϚϯυͰͷܭଌ
20. 20 ɾϥϯλΠϜίϚϯυ௚઀࣮ߦͰ͸͢΂ͯͷϥϯλΠϜΛܭଌͰ͖ͳ ͔ͬͨͷͰҧ͏ΞϓϩʔνͰܭଌ ɾ͜͜Ͱ΋Nabla Containersͷนཱ͕ͪ͸͔ͩΔ ɾଞͷϥϯλΠϜ͸Shim API v2ʹରԠ͍ͯ͠Δ ɾctr run —runtime=io.containerd.kata.v2 Έ͍ͨʹ࣮ߦͰ͖Δ ɾrunnc͸Shim API v2ʹରԠ͍ͯ͠ͳ͍ containerdͷctrίϚϯυͰͷܭଌ
21. 21 ɾ/etc/containerd/config.toml ɾctr run --runtime io.containerd.runtime.v1.linux Ͱ࣮ߦ ผͷํ๏Ͱ࣮ߦΛࢼΈΔ [plugins] [plugins.linux] shim = "containerd-shim" runtime = "/usr/local/bin/runnc"
22. 22 ɾctr: OCI runtime create failed: runnc did not terminate sucessfully: unknown ɾrunnc͕panic: Insufficient uniqueness in IDΛు͍ͯΔ ɾཁ͢Δʹίϯςφ໊͕୹͍ ɾϩάʹ͸͜Ε͕ݟ͋ͨΒͳ͍ͷͰΘ͔Γʹ͍͘ ɾERR: could not create tapabcdefg12345: no master interface: Link not found ɾίϯςφ໊Λ௕͘͢Δͱࠓ౓͸͜ͷΤϥʔ ɾ͜ΕҎ্͸·ͩௐࠪͰ͖͍ͯͳ͍ ࣮ߦ݁Ռ
23. dockerίϚϯυͰͷܭଌ
24. 24 ɾϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌɺctrίϚϯυͰͷܭଌɺͱ΋ ʹ͢΂ͯͷϥϯλΠϜΛܭଌ͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ ɾͷͰ࣍͸dockerίϚϯυͰτϥΠ dockerίϚϯυͰͷܭଌ
25. 25 ɾFirecrackerͷಈ͔͠ํ͚ͩΘ͔ΒΜɺͱࢥͬͨΒudzura͞ΜʹΑΔ φΠεࢿྉ͕ ɾhttps://speakerdeck.com/udzura/firecracker-from-low-layer-to- hight?slide=14 ɾKata ContainersͰFirecrackerΛಈ͔͢kata-fcΛར༻ ɾhttps://github.com/kata-containers/documentation/wiki/Initial- release-of-Kata-Containers-with-Firecracker-support ɾDockerͷdevicemapperαϙʔτ͕ඞཁ͕ͩɺݱࡏ࠷৽ͷ18.09͕ devicemapperରԠ͍ͯ͠ͳ͍ͷͰɺ18.06Λར༻͢Δඞཁ͋Γ Docker + Firecraker
26. 26 ɾ࠷ॳ͸ҙຯ͕Θ͔Βͳ͔ͬͨ ɾ͑ɺͲͬͪ΋OCIϥϯλΠϜ͡Όͳ͍ͷʁ ɾFiracracker͸ϚΫϩͳࢹ఺ͰݟΔͱOCIϥϯλΠϜͱͯ͠ݟΔ͜ͱ ΋Ͱ͖Δ͕ϛΫϩͳࢹ఺ͰݟΔͱVMM ɾKata Containers͸VM಺ͰίϯςφΛىಈ͢ΔΞϓϩʔνͷOCIϥ ϯλΠϜ ɾVMMͱͯ͠σϑΥϧτͰQEMUΛར༻͢Δ͕ࠩ͠ସ͑Մೳ ɾͭ·ΓKata ConͷVMMΛFirecrackerʹࠩ͠ସ͑Δ͜ͱ͕Ͱ͖Δ Kata Containers + Firecracker?
27. ctrͱdockerͰFirecrackerͷ ىಈ͕࣌ؒ૝ఆͱҟͳΔ
28. 28 ɾctr: real 0m6.320s ɾdocker: real 0m4.105s ɾdockerͷํ͕ɺdockerdΛܦ༝͢Δ෼஗͘ͳΓͦ͏ͳͷʹͳͥʁ ɾctr͸naive snapshotterΛར༻ ɾdocker͸devicemapperΛར༻ ɾctrͰdevmapper snapshotterΛར༻͢Ε͹ಉ͡৚݅ͰൺֱͰ͖ͦ͏ ɾ→ ະணख ctrͱdockerͰͷFirecrackerىಈ࣌ؒ
29. 1. ίετ
30. 30 ɾݕূ؀ڥΛVagrant+VirtualBoxͰߏங ɾKataͱFirecracker͸KVM͕ඞཁ ɾVirtualBoxͰ͸KVMಈ͔ͳ͍ ɾVMWare Fusion + Vagrant VMWare ProviderΛߪೖ ɾVMWare Fusion: 9,925ԁ ɾVagrant VMWare Provider: $79 per seat ׂͱ͓͕͔͔ۚΔ (on macOS)
31. 31 ɾVagrant + VMWare FusionͰmodprobe vhost_vsock͕Τϥʔʹ ͳͬͯ͠·͏ͷͰAWS EC2্Ͱ΋ݕূ ɾKVMΛಈ͔ͨ͢ΊʹϕΞϝλϧΠϯελϯε͕ඞཁ ɾi3.metalͰ4.992USD/࣌ؒ ɾ1೔ͰBilling AlertඈΜͰདྷͨ ׂͱ͓͕͔͔ۚΔ (on AWS)
32. 1. ࢀߟࢿྉ
33. 33 ɾhttps://github.com/mizzy/container-playground ɾmeasurements/ ɾVagrant + VMWare FusionͰಈ͔ͯ͠Δ΍ͭ ɾcompare_on_i3_metal/ ɾVagrant + AWS EC2 i3.metalΠϯελϯεͰಈ͔ͯ͠Δ΍ͭ ɾ੔ཧͰ͖ͯͳ͍͠ɺ௨͠Ͱvagrant provisionͯ͠ͳ͍ͷͰಈ͔ͳ ͍ͱ͜Ζ͋Γͦ͏ ɾࢼͯ͠ΈͯΘ͔Βͳ͍͜ͱ͕͋Ε͹ԿͰ΋ฉ͍͍ͯͩ͘͞ ܭଌ༻ϦϙδτϦ
34. 34 ɾࠓ࿩୊ͷ͍Ζ͍ΖͳίϯςφϥϯλΠϜΛൺֱͯ͠Έͨ ɾhttps://www.slideshare.net/KoheiTokunaga/ss-123664087 ɾ֤छϥϯλΠϜͷಛ௃΍ൺֱͳͲͱͯ΋ࢀߟʹͳΔ ɾNabla ContainersΛಈ͔͢ʹ͋ͨͬͯͱͯ΋ࢀߟʹͳͬͨ ɾࢿྉͰ͸ܭଌʹkubernetes-sigs/cri-toolsΛར༻͍ͯ͠ΔͷͰ͜Ε ΋ࢼͯ͠Έ͍ͨ ࢀߟࢿྉ