Past, Present and Future of Containers

At Dosudo 09/26/15

Transcript

1. Past, Present and Future of Containers Timothy Chen
2. About me: - Lead Distributed Systems Engineer at Mesosphere - Apache Mesos, Drill PMC - Maintains Spark on Mesos, Docker Swarm
3. Outline: Why containers? What is a container? How is it implemented now? Where is it going?
6. VMs
9. The Matrix From Hell django web frontend ? ? ? ? ? ? node.js async API ? ? ? ? ? ? background workers ? ? ? ? ? ? SQL database ? ? ? ? ? ? distributed DB, big data ? ? ? ? ? ? message queue ? ? ? ? ? ? my laptop your laptop QA staging prod on cloud VM prod on bare metal
10. Another Matrix from Hell ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
11. Solution: the intermodal shipping container
12. Solved!
13. Solution to the deployment problem: the Linux container
14. Linux containers... Units of software delivery (ship it!) • run everywhere – regardless of kernel version – regardless of host distro – (but container and host architecture must match*) • run anything – if it can run on the host, it can run in the container – i.e., if it can run on a Linux kernel, it can run *Unless you emulate CPU with qemu and binfmt
15. CFO, CIO, CTO, ... • LESS overhead! • MOAR consolidation! • MOAR agility! • LESS costs!
18. Solaris Zones 2002
22. High level approach: it's a lightweight VM • own process space • own network interface • can run stuff as root • can have its own /sbin/init (different from the host)
23. Low level approach: it's chroot on steroids • can also not have its own /sbin/init • container = isolated process(es) • share kernel with host • no device emulation (neither HVM nor PV)
24. Separation of concerns: Dave the Developer • inside my container: – my code – my libraries – my package manager – my app – my data
25. Separation of concerns: Oscar the Ops guy • outside the container: – logging – remote access – network configuration – monitoring
26. How does it work? Isolation with namespaces • pid • mnt • net • uts • ipc • user
27. How does it work? Isolation with cgroups • memory • cpu • blkio • devices
28. Efficiency: almost no overhead • processes are isolated, but run straight on the host • CPU performance = native performance • memory performance = a few % shaved off for (optional) accounting • network performance = small overhead; can be optimized to zero overhead
30. https://www.youtube.com/watch?v=JHqM_5X3MBU
31. Docker
32. What's Docker? • Open Source engine to commoditize LXC • using copy-on-write for quick provisioning • allowing to create and share images • propose a standard format for containers
33. Yes, but... • « I don't need Docker; I can do all that stuff with LXC tools, rsync, some scripts! » • correct on all accounts; but it's also true for apt, dpkg, rpm, yum, etc. • the whole point is to commoditize, i.e. make it ridiculously easy to use
34. Docker: authoring images • you can author « images » – either with « run+commit » cycles, taking snapshots – or with a Dockerfile (=source code for a container) – both ways, it's ridiculously easy • you can run them – anywhere – multiple times
37. Dockerfile example FROM ubuntu RUN apt-get -y update RUN apt-get install -y g++ RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ... RUN apt-get install -y libmozjs185-dev libicu-dev libtool ... RUN apt-get install -y make wget RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf- RUN cd /tmp/apache-couchdb-* && ./configure && make install RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini EXPOSE 8101 CMD ["/usr/local/bin/couchdb"]
38. Docker: sharing images • you can push/pull images to/from a registry (public or private) • you can search images through a public index • dotCloud maintains a collection of base images (Ubuntu, Fedora...) • satisfaction guaranteed or your money back
39. Docker: not sharing images • private registry – for proprietary code – or security credentials – or fast local access
42. Efficiency: storage-friendly • unioning filesystems (AUFS, overlayfs) • snapshotting filesystems (BTRFS, ZFS) • copy-on-write (thin snapshots with LVM or device-mapper) This wasn't part of LXC at first; but you definitely want it!
43. Efficiency: storage-friendly • provisioning now takes a few milliseconds • … and a few kilobytes • creating a new base/image/whateveryoucallit takes a few seconds