Real World Docker for the Rubyist

Docker’s gotten a lot of press, but how does it fare in the real world Rubyists inhabit every day?

Together we’ll take a deep dive into how a real company transformed itself to run on Docker. We’ll see how to build and maintain Docker images tailored for Ruby. We’ll dig into proper configuration and deployment options for containerized applications. Along the way we’ll highlight the pitfalls, bugs and gotchas that come with such a young, fast moving platform like Docker.

Whether you’re in production with Docker or just dabbling, come learn how Docker and Ruby make an awesome combination.

Transcript

1. Real World Docker for the Rubyist @jasonrclark
2. I'm Jane, a new New Relic with a cool new feature to deploy! I'm Jill, an experienced Relic, ready to help
3. This document and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission. Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,” “expects” or words of similar import. Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov. New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.
4. Why Docker? Packaging!
6. 6 image container docker run
7. 7 image container docker run class instance new()
8. Why Docker? Isolation!
9. 9 server /app /other-app
10. 10 server /app /other-app container container
11. How do I get started? Install Docker Toolkit and write your Dockerfile
12. 12 https://www.docker.com/products/docker-toolbox
13. 13 FROM newrelic/base-builder loc-service/Dockerfile
14. 14 FROM ruby:2.3 WORKDIR /data/app COPY Gemfile Gemfile.lock /data/app RUN bundle install --deployment ADD . /data/app CMD ["unicorn", "--config", "..."] Dockerfile ?
15. Where's the rest of my Dockerfile? Let's deploy your service first!
16. $terminal >>> 16 $ docker build -t loc-service .
17. $terminal >>> 17 $ docker build -t loc-service . Sending build context to Docker daemon Step 1 : FROM newrelic/base-builder 2.3: Pulling from newrelic/base-builder ... ---> 7ca70eb2dfea Step 2 : COPY . /data/app ---> 9f54fdf6b489 Removing intermediate container 669fd221 ... Step 14 : CMD supervisord Removing intermediate container 5247f58d Successfully built 1773de8387dc $
18. $terminal >>> 18 $ docker images REPOSITORY TAG IMAGE ID CREATED loc-service latest sha256:1773d 4 min ago
19. How do I get the image off my machine? Good question!
20. 20 dogestry
21. How do I get my containers started? Centurion!
22. $terminal >>> 22 $ docker run loc-service CRIT Supervisor running as root (no user in config file) WARN Included extra file "/etc/supervisord.d/app.conf" during parsing WARN Included extra file "/etc/supervisord.d/nginx.conf" during parsing WARN Included extra file "/etc/supervisord.d/ permissions.conf" during parsing INFO supervisord started with pid 1 INFO spawned: 'fix-permissions' with pid 8 INFO spawned: 'web-unicorn' with pid 9 INFO spawned: 'nginx' with pid 10 ...
25. 25 namespace :environment do desc 'Staging environment' task :staging => :common do set :image, 'newrelic/loc-service' host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
26. 26 namespace :environment do desc 'Staging environment' task :staging do set :image, 'newrelic/loc-service' host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
27. 27 namespace :environment do desc 'Staging environment' task :staging do set :image, 'newrelic/loc-service' host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
28. 28 namespace :environment do desc 'Staging environment' task :staging do set :image, 'newrelic/loc-service' host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
29. $terminal >>> 29 $ gem install centurion
30. $terminal >>> 30 $ centurion -e staging \ -p loc-service \ -a deploy
31. Ready? Go!
32. $terminal >>> 32 $ centurion -e staging \ -p loc-service \ -a deploy ...Fetching image loc-service:latest ...Using CLI to pull latest -- Connecting to staging-1.nr-internal.net RESOLVED loc-service:latest => 999cd48e588d ** Invoke deploy:verify_image (first_time) ** Execute deploy:verify_image -- Connecting to staging-1.nr-internal.net .... Found container up for 6 seconds Container is up!
33. $terminal >>> 33 $ centurion -e staging \ -p loc-service \ -a deploy ...Fetching image loc-service:latest ...Using CLI to pull latest -- Connecting to staging-1.nr-internal.net RESOLVED loc-service:latest => 999cd48e588d ** Invoke deploy:verify_image (first_time) ** Execute deploy:verify_image -- Connecting to staging-2.nr-internal.net .... Found container up for 6 seconds Container is up!
34. $terminal >>> 34 $ centurion -e staging \ -p loc-service \ -a deploy ...Fetching image loc-service:latest ...Using CLI to pull latest -- Connecting to staging-1.nr-internal.net RESOLVED loc-service:latest => 999cd48e588d ** Invoke deploy:verify_image (first_time) ** Execute deploy:verify_image -- Connecting to staging-2.nr-internal.net .... Found container up for 6 seconds Container is up!
37. WHERE'S MY APP!?!?!
38. Yes, but how does that help? Are your logs going to STDOUT?
39. 39 # STDOUT or get out! Rails.logger = Logger.new(STDOUT) config/environments/production.rb
41. How? Let's take a closer look on the container
42. $terminal >>> 42 $ docker -H tcp://staging-1.nr-internal.net:2375 ps CONTAINER ID IMAGE COMMAND 5189764ce084 loc-service "supervisord"
43. $terminal >>> 43 $ docker -H tcp://staging-1.nr-internal.net:2375 ps CONTAINER ID IMAGE COMMAND 5189764ce084 loc-service "supervisord" $ docker -H tcp://staging-1.nr-internal.net:2375 \ exec -it 5189764ce084 bash [root@5189764ce084 /]#
44. About 300MB How much memory does your app use?
45. 45 namespace :environment do desc 'Staging environment' task :staging do set :image, 'newrelic/loc-service' memory 2.gigabytes host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
46. $terminal >>> 46 $ docker run -m 2g loc-service
47. How do I get more Unicorn workers? Try the ENV!
48. $terminal >>> 48 $ docker run -it -e UNICORN_WORKERS=3 loc-service bash [root@5189764ce084 /]# env ... UNICORN_WORKERS=3
49. 49 ... # how many worker processes to start worker_processes ENV["UNICORN_WORKERS"].to_i unicorn_config.rb
50. 50 namespace :environment do task :common do ... env_vars UNICORN_WORKERS: 3 end end centurion-config/loc-service.rake
51. 51 mysql: &mysql encoding: utf8 adapter: <%=ENV['DB_ADAPTER']||'mysql2'%> username:<%=ENV['DB_USERNAME']||'root'%> host: <%=ENV['DB_HOST']||'localhost'%> password:'<%=ENV['DB_PASSWORD']%>' database:<%=ENV['DB_SCHEMA']%> config/database.yml
52. 52 module MyRailsApp class Application < Rails::Application config.service_url = (ENV['SERVICE_URL'] || 'http://...') config.service_timeout = (ENV['SERVICE_TIMEOUT'] || 5).to_f end end config/application.rb
53. 53 class LinesOfCodeController def index fetch(Rails.configuration.service_url, Rails.configuration.service_timeout) end end app/controllers/loc_controller.rb
55. That one-line Dockerfile... how's that work? Let's take a look!
57. 57 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 base-builder/Dockerfile
58. 58 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 # Install supervisord... RUN easy_install supervisor # Install nginx... RUN yum install -y nginx base-builder/Dockerfile
59. 59 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 RUN gem install bundler RUN rbenv rehash ... base-builder/Dockerfile
60. 60 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 RUN gem install bundler RUN rbenv rehash RUN bundle install ... base-builder/Dockerfile
61. $terminal >>> 61 $ docker build -t newrelic/base-builder . ... Step 3 : RUN bundle install ---> Running in cb91de794b44 Could not locate Gemfile
62. 62 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 RUN gem install bundler RUN rbenv rehash ONBUILD COPY Gemfile* . ONBUILD RUN bundle install base-builder/Dockerfile
63. 63 FROM newrelic/base-builder RUN use-puma RUN try-python another-app/Dockerfile
64. 64 #! /bin/sh cp /etc/templates/web.puma.conf \ /etc/supervisord.d/app.conf cp /etc/templates/puma_config.rb \ /data/app/config/puma_config.rb base-builder/usr/bin/use-puma
65. 65 FROM newrelic/base-builder RUN use-puma RUN try-python another-app/Dockerfile !!!
66. I'm getting an error writing a file What's the error?
67. $terminal >>> 67 Errno::EACCES: Permission denied @ rb_sysopen - /just-a-file-nothing-to-see-here.txt
68. With who? You've got a problem with nobody
69. nobody 69
70. 70 [program:web-unicorn] command=bundle exec unicorn -c config/unicorn_config.rb autostart=true autorestart=true directory=/data/app user=nobody stdout_logfile = /dev/stdout stdout_logfile_maxbytes = 0 stderr_logfile = /dev/stderr stderr_logfile_maxbytes = 0 /etc/supervisord.d/web-unicorn.conf
71. If Docker runs as root, why don't we? Defense in depth!
72. Can I run my tests in Docker? There are options...
73. $terminal >>> 73 $ docker run loc-service bundle exec rake /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ... Randomized with seed 22351 ................. Finished in 0.23058 seconds (files took 2.86 seconds to load) 17 examples, 0 failures Randomized with seed 22351 $
74. $terminal >>> 74 $ docker build -t loc-service . && \ docker run loc-service bundle exec rake docker build . Sending build context to Docker daemon... Step 1 : FROM newrelic/base-builder # Executing 12 build triggers... ... ... ... /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ... ...
75. $terminal >>> 75 $ docker build -t loc-service . && \ docker run loc-service bundle exec rake docker build . Sending build context to Docker daemon... Step 1 : FROM newrelic/base-builder # Executing 12 build triggers... ... ... ... /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ... ...
76. $terminal >>> 76 $ docker run -it \ -v /src/my-app:/test-app \ loc-image \ "cd /test-app && bundle exec rake"
77. I need a Redis instance We've got just the thing
78. 78 namespace :environment do desc 'Staging environment' task :staging do set :image, 'newrelic/redis' memory 1.gigabytes host 'staging-3.nr-internal.net' env_vars REDIS_MAXMEMORY: '1G' env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW' end centurion-config/loc-redis.rake
79. 79 namespace :environment do desc 'Staging environment' task :staging do set :image, 'newrelic/redis' memory 1.gigabytes host 'staging-3.nr-internal.net' env_vars REDIS_MAXMEMORY: '1G' env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW' end centurion-config/loc-redis.rake
80. 80 namespace :environment do desc 'Staging environment' task :staging do set :image, 'newrelic/redis' memory 1.gigabytes host 'staging-3.nr-internal.net' env_vars REDIS_MAXMEMORY: '1G' env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW' end centurion-config/loc-redis.rake
81. Centurion's cool, but what else is out there? A lot!
85. deploying with centurion ENV-based configuration building shared images security testing the future thanks! @jasonrclark