Adopting Docker

Talk from Pragmatic Docker, all about use cases for adopting docker from an operations perspective.

Transcript

1. Adopting Docker Puppet Labs Gareth Rushgrove Common use cases, from getting started to production
2. @garethr
3. Gareth Rushgrove
4. Gareth Rushgrove
5. This talk
6. Solving local development problems Gareth Rushgrove
7. Improving continuous integration Gareth Rushgrove
8. First steps into production Gareth Rushgrove
9. Going full container Gareth Rushgrove
10. Local development dependencies
11. Developers want to run different operating systems, and manage them adhoc Gareth Rushgrove
12. Local operating systems are often different to the target production operating system Gareth Rushgrove
13. Modern software stacks are complex Gareth Rushgrove
14. Not everyone can know how to manage every component Gareth Rushgrove
15. Being able to run everything locally is incredibly handy Gareth Rushgrove
16. Local virtual machines help, but still have a maintenance cost Gareth Rushgrove
17. Gareth Rushgrove application code dependent services a computer
18. Gareth Rushgrove application code dependent services a shared computer application code application code
19. Gareth Rushgrove application code services in virtual machines a computer
20. Gareth Rushgrove application code dependent services in containers a computer container runtime in a virtual machine
21. Gareth Rushgrove application code services in containers a computer with a native container runtime
22. Gareth Rushgrove Docker Machine
23. $ docker-machine create -d virtualbox dev INFO[0000] Creating SSH key... INFO[0000] Creating VirtualBox VM... INFO[0007] Starting VirtualBox VM... INFO[0007] Waiting for VM to start... INFO[0041] "dev" has been created and is now the active machine. Gareth Rushgrove Create local container runtimes
24. Gareth Rushgrove $ docker-machine ls NAME ACTIVE DRIVER STATE URL dev * virtualbox Running tcp://192.168.99.127:2376
25. Gareth Rushgrove $ docker-machine create -d digitalocean remote INFO[0000] Creating SSH key... INFO[0001] Creating Digital Ocean droplet... INFO[0002] Waiting for SSH... INFO[0070] Configuring Machine... INFO[0109] "remote" has been created and is now the active machine. Create remote container runtimes
26. Gareth Rushgrove $ docker-machine ls NAME ACTIVE DRIVER STATE URL dev virtualbox Running tcp://192.168.99.127:2376 remote * digitalocean Running tcp://104.236.253.181:2376
27. Gareth Rushgrove Docker Compose
28. Gareth Rushgrove $ cat docker-compose.yml web: build: . links: - db - redis ports: - "8000:8000" db: image: postgres image: redis Share workspace definitions
29. Gareth Rushgrove $ cat docker-compose.yml web: build: . links: - db - redis - elasticsearch ports: - "8000:8000" db: image: postgres image: redis image: elasticsearch Easily add new services
30. Continuous Integration
31. Everyone wants faster build times Gareth Rushgrove
32. Isolated builds are essential to avoid unrelated failures due to environment pollution Gareth Rushgrove
33. Virtual machines solve the isolation, but add overhead to a test run Gareth Rushgrove
34. Lots of hardware capacity for peak load can lead to utilisation issues Gareth Rushgrove
35. Docker provides just enough isolation for many test use cases Gareth Rushgrove
36. And a scheduler like Mesos can result in best possible utilisation Gareth Rushgrove
37. Gareth Rushgrove Docker Jenkins plugin
38. The aim of the docker plugin is to be able to use a docker host to dynamically provision a slave, run a single build, then tear-down that slave. Gareth Rushgrove https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin
39. Gareth Rushgrove Mesos Jenkins plugin
40. The mesos-jenkins plugin allows Jenkins to dynamically launch Jenkins slaves on a Mesos cluster depending on the workload! Gareth Rushgrove https://wiki.jenkins-ci.org/display/JENKINS/Mesos+Plugin
41. Gareth Rushgrove jenkins slave jenkins slave jenkins slave jenkins master Job
42. Gareth Rushgrove jenkins slave jenkins slave jenkins slave jenkins master Job
43. Gareth Rushgrove jenkins slave jenkins slave jenkins slave jenkins master Job Job Job one job per slave is no problem
44. Gareth Rushgrove jenkins slave jenkins slave jenkins slave jenkins master Job Job Job but with multiple jobs per slave we can run into dependency issues
45. Gareth Rushgrove mesos slave mesos slave mesos slave jenkins master Job Job Job each container is a short lived jenkins slave
46. Gareth Rushgrove eBay CI Solution
47. Docker is an implementation detail. You’re not directly using the Docker interfaces. Gareth Rushgrove
48. A package format and a runtime
49. 1 container per VM pattern Gareth Rushgrove
50. Gareth Rushgrove
51. It’s all about security domains Gareth Rushgrove
52. Take advantage of containers in production now, while minimising downsides Gareth Rushgrove
53. Operating containers requires new knowledge and new tools Gareth Rushgrove
54. Visibility for operations Gareth Rushgrove
55. Problems with ps Gareth Rushgrove
56. Gareth Rushgrove $ ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND ... 999 1807 0.2 11.4 867624 464572 ? Ssl 09:38 0:21 mysqld Is this process in a container?
57. Gareth Rushgrove $ ps -eo ucmd,cgroup COMMAND CGROUP ... mysqld 9:perf_event:/docker/61e76d2c39121282474ff895b9b3ba2addd775cdea6d2ba89ce76c28 Which container is that?
58. Gareth Rushgrove Sysdig
59. Provides a Kernel module, which hooks into cgroups and namespaces Gareth Rushgrove (Prediction: We’ll see more of this kind of thing)
60. Gareth Rushgrove $ sudo sysdig -c topcontainers_cpu CPU% container.name ----------------------------------------------------------------------- 90.13% mysql 15.93% wordpress1 7.27% haproxy 3.46% wordpress2 CPU usage acros containers
61. Gareth Rushgrove $ sudo sysdig -pc -c topprocs_cpu container.name=client CPU% Process container.name ---------------------------------------------- 02.69% bash client 31.04% curl client 0.74% sleep client CPU usage in a single container
62. Gareth Rushgrove $ sudo sysdig -pc -c topprocs_net Bytes Process Host_pid Container_pid container.name --------------------------------------------------------------- 72.06KB haproxy 7385 13 haproxy 56.96KB docker.io 1775 7039 host 44.45KB mysqld 6995 91 mysql 44.45KB mysqld 6995 99 mysql 29.36KB apache2 7893 124 wordpress1 29.36KB apache2 26895 126 wordpress4 29.36KB apache2 26622 131 wordpress2 29.36KB apache2 27935 132 wordpress3 29.36KB apache2 27306 125 wordpress4 22.23KB mysqld 6995 90 mysqlclient Network bandwidth
63. Gareth Rushgrove $ sudo sysdig -pc -A -c echo_fds "fd.ip=172.17.0.3 and fd.ip=172.17.0.7" ------ Write 103B to [haproxy] [d468ee81543a] 172.17.0.7:37557->172.17.0.3:80 (hapr GET / HTTP/1.1 User-Agent: curl/7.35.0 Host: 172.17.0.7 Accept: */* X-Forwarded-For: 172.17.0.8 ------ Read 103B from [wordpress1] [12b8c6a04031] 172.17.0.7:37557->172.17.0.3:80 ( GET / HTTP/1.1 User-Agent: curl/7.35.0 Host: 172.17.0.7 Accept: */* X-Forwarded-For: 172.17.0.8 ------ Write 346B to [wordpress1] [12b8c6a04031] 172.17.0.7:37557->172.17.0.3:80 (a HTTP/1.1 302 Found Date: Sat, 21 Feb 2015 22:19:18 GMT Traffic between containers
64. Metadata Gareth Rushgrove
65. Gareth Rushgrove
66. Metadata like name, version, build time, build host, dependencies, descriptions, licence, signature and urls Built in logic like pre/post install scripts An API to interact with this – the rpm or apt/deb commands Gareth Rushgrove - - - https://www.devco.net/archives/2015/03/30/some-thoughts-on-operating-containers.php
67. Gareth Rushgrove $ docker exec -ti mycontainer container --metadata|json_reformat { "validate_method": "/srv/support/bin/validate.sh", "start_method": "/srv/support/bin/start.sh", "update_method": "/srv/support/bin/update.sh" "validate": true, "build_cause": "TIMERTRIGGER", "build_tag": "jenkins-docker rbldnsd-55", "ci": true, "image_tag_names": [ "hub.my.net/ripienaar/rbldnsd" ], "project": "rbldnsd", "build_time": "2015-03-30 06:02:10", "build_time_stamp": 1427691730, "image_name": "ripienaar/rbldnsd", "gitref": "e1b0a445744fec5e584919711cafd8f4cebdee0e", }
68. $ docker exec -ti rbldnsd container --examine Container first started at 2015-03-30 05:02:37 +0000 (1427691757) Names: Project Name: centos_base Image Name: ripienaar/centos_base Image Tag Names: hub.my.net/ripienaar/centos_base Build Info: CI Run: true Git Hash: fcb5f3c664b293c7a196c9809a33714427804d40 Build Cause: TIMERTRIGGER Build Time: 2015-03-24 03:25:01 (1427167501) Build Tag: jenkins-docker centos_base-20 Actions: START: not set UPDATE: not set VALIDATE: not set Gareth Rushgrove
69. Community standards here would be awesome Gareth Rushgrove
70. Base image Gareth Rushgrove
71. Scratch images Minimal distros like busybox Full OS like Ubuntu or Debian Full OS with working init system Gareth Rushgrove - - - -
72. No one right answer. Depends on context; existing tooling, application runtime, etc. Gareth Rushgrove
73. Managing configuration Gareth Rushgrove
74. Separate configuration from containers Gareth Rushgrove
75. Retain advantages of declarative configuration management tools Gareth Rushgrove
76. Gareth Rushgrove
77. Gareth Rushgrove
78. Clusters of containers
79. Warning Beyond here lies the future Gareth Rushgrove
80. Higher level interfaces Multi-host scheduling Health checks Load balancing Gareth Rushgrove - - - -
81. Gareth Rushgrove Borg
82. Gareth Rushgrove Kubernetes
83. Gareth Rushgrove Mesos
84. Gareth Rushgrove CoreOS
85. Gareth Rushgrove
86. Gareth Rushgrove Lattice
87. Gareth Rushgrove Docker Swarm
88. Gareth Rushgrove Amazon ECS
89. A quest for an API Gareth Rushgrove
90. Current lack of tools build atop those APIs Gareth Rushgrove
91. Conclusions
92. We should talk about container use cases more than about containers Gareth Rushgrove
93. Different use cases probably require different tooling Gareth Rushgrove
94. Different use cases will have different best practices Gareth Rushgrove
95. You don’t have to use containers everywhere to see benefits Gareth Rushgrove
96. Questions? And thanks for listening Gareth Rushgrove