Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Wantedly から Chef を一掃した話 / #chibadan
Search
Daisuke Fujita
October 25, 2016
Programming
24
11k
Wantedly から Chef を一掃した話 / #chibadan
サービス開発の現場を語る! #chibadan の発表資料です
http://chibadan.connpass.com/event/41625/
Daisuke Fujita
October 25, 2016
Tweet
Share
More Decks by Daisuke Fujita
See All by Daisuke Fujita
SREcon19 Asia/Pacific Recap
dtan4
0
150
Our Practices of Delegating Ownership in Microservices World
dtan4
4
8.8k
Kubernetes Cluster Upgrade / Mercari Meetup for Microservices Platform
dtan4
3
4.4k
KubeCon EU 2018 Recap: Multi-Tenancy in Kubernetes: Best Practices Today, and Future Directions / Kubernetes Meetup Tokyo 11 #k8sjp
dtan4
1
1.8k
さようなら Chef こんにちは Dockerfile / Web Tech Tokyo #1
dtan4
6
7k
Docker をフル活用したインフラの紹介と成長し続けるためのインフラ戦略 / #abejameetup
dtan4
19
3.9k
Docker Compose PaaS の作り方、そして社内に導入した話 / #yapc8oji
dtan4
1
8.2k
Writing Kubenetes tools in Go
dtan4
1
3.5k
最近の wercker 便利って話 #tqrk10
dtan4
2
920
Other Decks in Programming
See All in Programming
RailsのPull requestsのレビューの時に私が考えていること
yahonda
5
1.7k
gopls を改造したら開発生産性が高まった
satorunooshie
8
240
CPython 인터프리터 구조 파헤치기 - PyCon Korea 24
kennethanceyer
0
240
PagerDuty を軸にした On-Call 構築と運用課題の解決 / PagerDuty Japan Community Meetup 4
horimislime
1
110
2万ページのSSG運用における工夫と注意点 / Vue Fes Japan 2024
chinen
3
1.3k
Tuning GraphQL on Rails
pyama86
2
1k
NSOutlineView何もわからん:( 前編 / I Don't Understand About NSOutlineView :( Pt. 1
usagimaru
0
110
Snowflake x dbtで作るセキュアでアジャイルなデータ基盤
tsoshiro
2
430
推し活の ハイトラフィックに立ち向かう Railsとアーキテクチャ - Kaigi on Rails 2024
falcon8823
6
2.2k
Identifying User Idenity
moro
6
7.9k
Streams APIとTCPフロー制御 / Web Streams API and TCP flow control
tasshi
1
290
CSC305 Lecture 13
javiergs
PRO
0
130
Featured
See All Featured
Designing for Performance
lara
604
68k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
42
9.2k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
3
370
The Power of CSS Pseudo Elements
geoffreycrofte
72
5.3k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
159
15k
How to train your dragon (web standard)
notwaldorf
88
5.7k
Being A Developer After 40
akosma
86
590k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
7
150
Fashionably flexible responsive web design (full day workshop)
malarkey
404
65k
Why You Should Never Use an ORM
jnunemaker
PRO
53
9k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
107
49k
Building Flexible Design Systems
yeseniaperezcruz
327
38k
Transcript
8BOUFEMZ͔Β $IFGΛҰͨ͠ 2016-10-25 αʔϏε։ൃͷݱΛޠΔʂ Daisuke Fujita (@dtan4) https://flic.kr/p/rU5bCU
Daisuke Fujita / @dtan4 Engineer, Infrastructure Team @ Wantedly, Inc.
AWS, Terraform / Terraforming, Developer Productivity
8BOUFEMZ͔Β $IFGΛҰͨͭͭ͠ %PDLFSपΓΛஔ͖͑ͨ 2016-10-25 αʔϏε։ൃͷݱΛޠΔʂ Daisuke Fujita (@dtan4) https://flic.kr/p/rU5bCU
Infrastructure at Wantedly AWS + DNSimple શαʔϏε͕ Docker ίϯςφͰՔಇ EC2
Πϯελϯεͷ OS CoreOS ੲ Ubuntu + Chef Capistrano ΛͬͨಠࣗͷσϓϩΠγεςϜ + Heroku CLI ಉ༷ͷΠϯλʔϑΣΠεΛ࣋ͭ CLI https://speakerdeck.com/dtan4/number-abejameetup
Web (Rails) Docker image 2ஈߏ ࠩͷΈͷ fetch + bundle install
Ͱ σϓϩΠΛߴԽ intermediate Chef + Packer ΠϝʔδαΠζ 3GB bundle install git clone wantedly/wantedly Ruby FROM Ubuntu 14.04 rake assets:precompile bundle install git checkout <branch> git fetch FROM intermediate web (Dockerfile) intermediate (Chef)
Web (Rails) Docker image 2ஈߏ ࠩͷΈͷ fetch + bundle install
Ͱ σϓϩΠΛߴԽ intermediate Chef + Packer ΠϝʔδαΠζ 3GB bundle install git clone wantedly/wantedly Ruby FROM Ubuntu 14.04 rake assets:precompile bundle install git checkout <branch> git fetch FROM intermediate web (Dockerfile) intermediate (Chef)
pull :intermediate push :production pull :production build :production $ sap
prod deploy (3:00 am) docker push :intermediate
Chef + Packer AWS Ҡߦॳ͔Β2Ҏ্Ҿ͖ܧ͕Ε͖ͯͨൿͷ Chef Ϩγϐ Packer Ͱ ubuntu:14.04
Πϝʔδ্ʹϓϩϏδϣχϯά => Private Registry ʹ push
Chef + Packer ??? ઐ༻ͷ EC2 Πϯελϯε1ͷΈͰՔಇ͍ͯͨ͠ ਓؒͷ Macbook ͰϓϩϏδϣχϯάͰ͖ͳ͘ͳ͍ͬͯͨ
Chef ʢͷΠϯετʔϧํ๏ʣͷਐԽʹैͦ͜͠Ͷͨ
Chef + Packer ??? 1Πϝʔδ࡞Δͷʹ1࣌ؒҎ্ Ͱ͖͕͋ͬͨΠϝʔδ 3GB Ruby ͷόʔδϣϯΞοϓ
apt ύοέʔδՃ͕໘ ΠϯϑϥνʔϜҎ֎͕৮Γʹ͍͘ σϓϩΠ࣌ͷ docker build ֤Πϯελϯεͷ͕͍
Chef + Packer => ??? ΠϝʔδαΠζΛͬͱখ͍ͨ͘͞͠… ଞͷϝϯόʔ৮Γ͍͢ߏʹ͍ͨ͠… σϓϩΠ͍ͨ͘͠… Dockerfile Ͱॻ͖͍ͨ…ʂ
Private Registry => ??? Docker Private Registry V1 (Python) ʹຊؾͰٽ͔͞Εͨ
Docker Distribution (Go) ͍ͩͿϚγʹͳ͚ͬͨͲ Registry ίϯςφ͕ࢮ͵ڪා૬มΘΒͣ ϚωʔδυαʔϏεʹҠߦ͍ͨ͠…ʂ ECR ౦ژʹདྷͨ͠
2016/10
2016/10 Wantedly, Inc. ͔Β Chef ΛҰ
2016/10 Chef + Packer -> Dockerfile + Docker Compose Ubuntu
14.04 -> Ubuntu 16.04, Alpine Linux 3.4 Private Docker Registry -> Amazon ECR Wantedly, Inc. ͔Β Chef ΛҰ
(3:00 am) docker push :intermediate pull :intermediate push :production pull
:production build :production $ sap prod deploy
None
Dockerfile Chef Ϩγϐ -> Dockerfile 1ຕʹू
Dockerfile αʔϏε͝ͱʹ1ຕͷ Dockerfile FROM ubuntu 16.04 / alpine:3.4 Dockerfile ϕετϓϥΫςΟε
Λ࣮ફͯ͠μΠΤοτ
Dockerfile Ruby, Node.js ͷΠϯετʔϧ ಉ͡ Dockerfile ʹॻ͘ ެࣜͷ -alpine Dockerfile
͔Βഈआ Alpine ͷόʔδϣϯ ࣗͨͪͰ੍ޚ αʔϏεʹ߹Θͤͨݴޠόʔδϣϯ
Dockerfile λΠϜελϯϓΛૠೖͯ͠ ҎԼຖճڧ੍తʹϏϧυ ιʔείʔυΛ git clone Access Token Λͬͨೝূ bundle
install -> assets:precompile ARG BUILD_DATE --build-args BUILD_DATE=$(date +%Y%m%d%H%M%S)
Docker Compose શΠϝʔδ Docker Compose ܦ༝Ͱ build & push build
args ΛҰׅཧ Ruby ͷόʔδϣϯΞοϓ ͔͜͜Β awesome: extends: base build: context: awesome args: - RUBY_MAJOR=2.3 - RUBY_VERSION=2.3.0 - RUBYGEMS_VERSION=2.6.6 - BUNDLER_VERSION=1.13.1 - NODE_VERSION=5.11.1 - NPM_VERSION=3.3.12 awesome-production: extends: awesome build: args: - RAILS_ENV=production image: ${AWS_ECR_DOMAIN}/awesome:intermediate
Alpine Linux busybox + α ͷܰྔσΟετϦ (~ 5MB) ύοέʔδϚωʔδϟ apk
͍͍ͨͯͳ͘ಈ͘ɻ ͳΜ͔Μ apk ʹϥΠϒϥϦҰ௨ΓἧͬͯΔɻ
Alpine Linux ʹҠߦ͢Δҙ ֎෦ίϚϯυݺͼग़͠ҙ `...` ͱ͔ os.Exec(...) ͱ͔ ݺͼग़͢όΠφϦ͕ Alpine
Linux ্ͩͱ ಈ͔ͳ͍Մೳੑ͕͋Δ (ex. wkhtmltopdf) ඪ४ίϚϯυ (ex. timeout) busybox ͳͷͰ Ubuntu ͷʹൺͯύϥϝʔλগͳ͔ͬͨΓ
None
Alpine Linux ʹҠߦ͢Δҙ όΠφϦΛ static link ͰϏϧυ͢Δ alpine-pkg-glibc, coreutils ΛೖΕΔ
֎෦ίϚϯυݺͼग़͠Ίͯ Pure ͳखஈʹ ͩΊͳΒૉʹ Ubuntu CentOS ͱ͍ͬͨ ϝδϟʔͳσΟετϦΛ͏ దʹαʔϏεΛఏڙͰ͖Δͷ͕͋͘·Ͱେલఏ
intermediate-build-server intermediate image Λఆظతʹ build ͢Δαʔό CoreOS on EC2 +
IAM Role ຖਂʹશ intermediate image Λ docker-compose build -> docker-compose push
intermediate-build-server #!/bin/bash set -eu metadata=$(curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/intermediate-build-server/) export AWS_ACCESS_KEY_ID=$(echo $metadata
| jq -r .AccessKeyId) export AWS_SECRET_ACCESS_KEY=$(echo $metadata | jq -r .SecretAccessKey) export AWS_SECURITY_TOKEN=$(echo $metadata | jq -r .Token) document=$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document) account_id=$(echo $document | jq -r .accountId) region=$(echo $document | jq -r .region) export AWS_ECR_DOMAIN=$account_id.dkr.ecr.$region.amazonaws.com export BUILD_DATE=$(date +%Y%m%d%H%M%S) cd intermediate-dockerfiles if [[ $# -eq 1 ]]; then /opt/bin/docker-compose build $1 else for app in $(cat docker-compose.yml | grep -E '^ [a-z-]+:' | grep -E -- '-(production|qa)' | sed -E 's/^ ([a-z-]*):/\1/'); do echo "[`date`] Building $app..." /opt/bin/docker-compose build $app done fi
intermediate-build-server ? ຊ CI as a Service ্ͰΓ͔ͨͬͨ image pull
-> build -> push Docker 1.10 ͔Β Docker image ͷΈ͕มΘͬͯɺ pull ͖ͯͨ͠Πϝʔδͷʮ్த·ͰΛʯΩϟογϡͱͯ͠ ͏͜ͱ͕Ͱ͖ͳ͘ͳͬͨ https://docs.docker.com/engine/userguide/storagedriver/ imagesandcontainers/#/content-addressable-storage
None
intermediate-build-server Docker 1.13 Ͱղফ༧ఆ
ECR AWS ECS ଐͷ ϑϧϚωʔδυ Docker Registry Terraform ͰϦϙδτϦΛཧ ϩάΠϯ͢ΔεΫϦϓτΛ
1͓͖࣌ؒʹఆظ࣮ߦ
ECR 1ϨϙδτϦ࠷େ1000Πϝʔδ 80 - 90Πϝʔδ / week ϖʔεͰ૿Ճ Lambda Ͱλά͍ͯͳ͍ΠϝʔδΛఆظతʹ͓আ
ECR dtan4/ecr-gc
Ҡߦ݁Ռ
Ҡߦ݁Ռ ΠϝʔδαΠζ 3.307 GB -> 2.24 GB -> 2.43 GB
Ϗϧυ࣌ؒ 48 m -> 16 m 5 s Alpine Linux Ubuntu
TODO Serverspec ॻ͍͚ͨͲ CI ͕ͳ͍ shallow clone ͰΠϝʔδΛ͞ΒʹμΠΤοτ ͱ͔ͱ͔…
Recap
None