Django and Docker: A Marriage made in Heaven

Ken Cochrane's Talks about Docker at DjangoCon US 2013 in Chicago. September 3rd 2013

Transcript

1. Django and Docker: a marriage made in heaven Ken Cochrane DjangoCon US 2013 1
2. About me • Ken Cochrane (@KenCochrane) • Engineer at dotCloud (Corporate sponsor of Docker) • Work on Docker and Docker related projects (index, registry, docs, etc) • Django user since 0.96.1 2
3. Outline • Intro to Docker • Common use cases • Demo • Questions 3
4. Quick survey 4
5. Quick survey • How many people have heard of Docker before today? 4
6. Quick survey • How many people have heard of Docker before today? • How many people have tried Docker? 4
7. Quick survey • How many people have heard of Docker before today? • How many people have tried Docker? • How many people are using Docker on a project today? 4
8. Where did Docker come from? • Docker is a rewrite of similar code that currently powers the dotCloud PaaS • Original version written in Python, new version written in Go. • Still a very young project, but mature for it’s age. 5
9. Docker Timeline • January 2013 Docker started as internal project inside of dotCloud • March 21, 2013 Solomon gives Docker lighting talk at PyCon US • March 27, 2013 Docker released to Public • June 2013 Docker adds OpenStack compatibility • August 2013 Docker 0.6 released 6
10. In the first 6 months • 5300+ GitHub stars • 125+ contributors • 50,000+ docker index pulls • 100’s of projects built on top of Docker • UI’s, mini-PaaS, remote desktop,etc • 1000’s of Dockerized applications 7
11. What is Docker? • Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere. 8
12. How does Docker work? • LinuX Containers (LXC) • Control Groups & Namespaces • AUFS 9
13. LinuX Containers (LXC) • lets you run a Linux system within another Linux system • A container is a group of processes on a Linux box, put together in an isolated environment • Inside the box, looks like a VM. • Outside of the box, it looks like normal processes. • chroot on steroids 10
14. Why Containers? • Speed: Boots in seconds. • Footprint: 100-1000 containers on one machine. Small disk requirements 11
15. Containers vs. VMs 12
16. Why are Containers lightweight? 13
17. Installing Docker 14
18. Docker Requirements • Linux Kernel 3.8 or above • AUFS • LXC • 64Bit 15
19. Installations • Server or Desktop (Vagrant) • Docker APT repo (get.docker.io) • Source code compile • Binary download • http://docs.docker.io/en/latest/installation/ 16
20. Vagrant install • git clone https://github.com/dotcloud/docker.git • vagrant up • vagrant ssh • sudo docker 17
21. Binary install • wget http://get.docker.io/builds/Linux/x86_64/docker-latest.tgz • tar -xf docker-latest.tgz • sudo ./docker -d & • Still need to install system startup script 18
22. Docker on Digital Ocean in 5 easy steps # Create Ubuntu 13.04 64 Bit Droplet, Then $ sudo apt-get update $ sudo apt-get install linux-image-extra-`uname -r` $ sudo sh -c "curl http://get.docker.io/gpg | apt-key add -" $ sudo sh -c "echo deb https://get.docker.io/ubuntu docker main > /etc/apt/ sources.list.d/docker.list" $ sudo apt-get update && sudo apt-get install lxc-docker 19
23. Docker on Digital Ocean in 1 step When creating the Droplet. Select the Docker image under the application 20
24. Awesome Sauce • The Awesome folks at Digital Ocean has given everyone here a great deal. • No more excuses! Try Docker for Free on Digital Ocean • Go to http://tinyurl.com/docker10 and enter in the promo code DJANGOCON2013 when prompted for a $10 credit. SSD backed VPS servers as low as $5/month Do it now, before the deal expires. • http://tinyurl.com/docker10 21
25. Use Cases 22
26. Docker Use Cases • Local Dev environment • Deployment • Unit testing • parallelize tests • one db per test • system tests 23
27. Testing 24
28. Unit testing • Use containers to isolate tests into their own environment. • No more worrying about tests not cleaning up after themselves. • Parallelize the tests across multiple machines 25
29. System Testing • Easily create all the different system configurations to test against • No need to worry about breaking or rebuilding a test server • Test Fabric scripts • http://agiliq.com/blog/2013/06/self-testing-fabfile-using-docker/ 26
30. Continuous Integration • Run unit tests after each source commit • StriderCD.com - Open source CI server • Uses Docker Containers to run CI tests before deployment. • TravisCI is also playing with Docker 27
31. Deployment 28
32. Docker Deployment • dokku • Flynn.io • deis.io • chef, puppet, salt, ansible, etc 29
33. Dokku • Open source PaaS • Docker powered mini-heroku • Less than 100 lines of Bash • Heroku buildpacks • Git push deployment • https://github.com/progrium/dokku 30
34. Flynn.io • Open source PaaS written in Go • Uses Docker to manage containers • One of the founders is author of Dokku • Still in development 31
35. deis.io • Open source written in Python • Git push deployments • Docker images, chef recipes or Heroku buildpacks • Supports scaling and application formation 32
36. Chef • chef-docker • docker-cookbook 33
37. Puppet garethr/docker 34 34
38. Salt stack • ipmb/salt-minion • kstaken/salt-test-runner 35
39. Ansible • cove/docker-ansible • bryfry/ansible-docker 36
40. Local Development 37
41. Local Dev Setup • VM’s are heavy, containers not so much • Run 100’s of containers on laptop vs a handful of VMs • Easier to duplicate production environment if you have a complex setup. • http://blog.scoutapp.com/articles/2013/08/28/docker-git-for-deployment 38
42. Cool Projects Using Docker 39
43. Cool projects • npmt.abru.pt: Node.js Module (NPM) testing • ptone/jiffylab: Zero configuration Python/Unix web based teaching environment • kitchen-Docker: Docker driver for Ruby’s test kitchen • MemcachedAsaService.com • Try RethinkDB, openstack-docker • Many many more. 40
44. npmt.abru.pt • Autonomously Testing All NPM modules • One container per module is created then destroyed when test is finished. • 39496 modules verified 41
45. kitchen-docker • A Ruby framework for running integration tests in an isolated environment • Uses Docker to run tests in containers • https://github.com/portertech/kitchen-docker 42
46. JiffyLab • Created by Preston Holmes • Provides an entirely web based environment for instruction. • Python and UNIX shell env running in it’s own Docker container • https://github.com/ptone/jiffylab 43
47. Memcached SaaS • Built as a class project • Memcached SaaS built on top of Docker • Built with Ruby on Rails • https://github.com/jbarbier/SaaS_Memcached/ 44
48. Try RethinkDB • SaaS that let you try out RethinkDB • One DB per container • Containers killed within 24 hours • 1000’s of containers on one host 45
49. openstack-docker • Deploy lightweight linux containers on Openstack • https://github.com/dotcloud/openstack- docker 46
50. Using Docker 47
51. Docker terms • Container: Linux container • Image: a snapshot of a container that when run creates a new container. • Index: Public docker image directory • Dockerfile: An automated script used to create an Image • Push/pull : Commands used to get images to and from the index • Run: Start a Docker image to create a running Container 48
52. Common commands • ps : lists the containers on the system • images : lists the images on the system • run : runs commands against an image to create a container • stop : stops a running command • build : builds a Dockerfile • inspect: shows you information about a container • pull: pulls down a new image from the docker index • logs: shows the logs for a given container 49
53. Docker Basics 50
54. Changes and Updates 51
55. Docker Run • Run command and exit • $ docker run ubuntu echo “hello world” • Start interactive shell in container • $ docker run -i -t ubuntu bash • Run command in background • $ docker run -d ubuntu /bin/sh -c “while true; do echo hello world; date; sleep 1; done” 52
56. Dockerfiles 53
57. Dockerfiles • Simple scripting language • Automate the creation of docker images • Built in caching • Add them to any project repo to Dockerize the project. • Online tutorial • http://www.docker.io/learn/dockerfile/ 54
58. Building Images by hand # start a ubuntu 12.10 container $ docker run -i -t ubuntu:12.10 bash # update the apt repo [a2bc13] $ apt-get update # install curl [a2bc13] $ apt-get install curl # exit out of container [a2bc13] $ exit # save changes to container as an image $ docker commit -m "comment" a2bc13 username/image 55
59. Build image using Dockerfile # curl image # VERSION 0.1 FROM ubuntu:12.10 MAINTAINER yourName name@example.com RUN apt-get -qq update RUN apt-get install -y curl 56
60. Docker build • docker build . • docker build - < Dockerfile • docker build github.com/creak/docker-firefox • use “-t” to tag built image • docker build -t myname/myimage . 57
61. Redis Dockerfile example # Redis # VERSION 0.1 FROM johncosta/redis MAINTAINER yourName name@example.com EXPOSE 6379 RUN mkdir /redis ENTRYPOINT ["/usr/bin/redis-server"] CMD ["--dir", "/redis"] 58
62. Docker Index 59
63. Docker index • Similar to PyPI but for Docker images • Public directory to store and download reusable docker images • Docker image meta data • Account required to publish images (free) • Written in Django • https://index.docker.io 60
64. Docker registry • Open source Python Flask app • https://github.com/dotcloud/docker-registry • Manages the storage of the images • Install private registry for private images 61
65. Docker API 62
66. Docker API • REST based API used to control the Docker daemon • live events api feed • some websocket support • The Docker CLI uses the same API • Clients available for most languages 63
67. Docker clients • python: docker-py • ruby: docker-ruby, docker-client, docker- api • javascript: docker-js, dockerui • Java: docker-java 64
68. Docker-py import docker # create client docker_client = docker.Client(base_url='unix://var/run/docker.sock', version="1.4") # create container container = docker_client.create_container('ubuntu', None, detach=True) container_id = container.get('Id') # start Container docker_client.start(container_id) # get Ip address meta = docker_client.inspect_container(container_id) ip = meta.get('NetworkSettings').get("IPAddress") # stop container docker_client.stop(container_id) 65
69. Docker UI’s • Shipyard: Django • https://github.com/ehazlett/shipyard • Docker-UI: Angular.js • https://github.com/crosbymichael/dockerui • Dockland: Ruby • https://github.com/dynport/dockland 66
70. Demo 67
71. Coming Soon 68
72. Docker 1.0 • Pluggable architecture • LXC, Solaris Zones, FreeBSD Zones, etc • AUFS, BTRFS, etc • Better OS support (RedHat, etc) 69
73. Docker 1.0 design 70
74. Want to Learn more? • Website: http://www.docker.io • Documentation: http://docs.docker.io • Github: https://github.com/dotcloud/docker • IRC: freenode #docker • Twitter: follow @docker • Google group: groups.google.com/forum/#!forum/docker- user • Meetups: Boston, New York, London, Paris, San Francisco, and more coming soon. Go to website for details. 71
75. Docker OpenSpace • Questions, installation help, etc • Today (Monday) @ 2-3pm • Grand suite 5 72
76. Docker Swag • I have a limited amount of Docker swag to give away. • T-shirts • Stickers • Send me a tweet @KenCochrane if you want one. 73
77. Questions? 74
78. Thank you! @KenCochrane 75