Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Better docker image+
Search
orisano
December 05, 2018
Technology
6
6.2k
Better docker image+
orisano
December 05, 2018
Tweet
Share
More Decks by orisano
See All by orisano
OSS Performance Tuning Tips
orisano
8
5.9k
Docker-Compose & BuildKit
orisano
4
1k
Container Build Talk
orisano
3
2.5k
dockerignore talk
orisano
2
7.2k
Socket.IO Introduction
orisano
0
3.2k
Profiling Go Application
orisano
11
7.9k
Multi-stage Builds Patterns & Practice
orisano
6
5.2k
better docker image
orisano
22
30k
the world of characters
orisano
8
1.4k
Other Decks in Technology
See All in Technology
開発と脆弱性と脆弱性診断についての話
su3158
1
1.1k
[CVPR2025論文読み会] Linguistics-aware Masked Image Modelingfor Self-supervised Scene Text Recognition
s_aiueo32
0
210
退屈なことはDevinにやらせよう〜〜Devin APIを使ったVisual Regression Testの自動追加〜
kawamataryo
1
130
RAID6 を楔形文字で組んで現代人を怖がらせましょう(実装編)
mimifuwa
0
300
MySQL HeatWave:サービス概要のご紹介
oracle4engineer
PRO
4
1.7k
人を動かすことについて考える
ichimichi
2
320
AIが住民向けコンシェルジュに?Amazon Connectと生成AIで実現する自治体AIエージェント!
yuyeah
0
260
Go で言うところのアレは TypeScript で言うとコレ / Kyoto.なんか #7
susisu
5
1.5k
歴代のWeb Speed Hackathonの出題から考えるデグレしないパフォーマンス改善
shuta13
6
600
Understanding Go GC #coefl_go_jp
bengo4com
0
1.1k
OpenAPIから画面生成に挑戦した話
koinunopochi
0
150
ソフトウェア エンジニアとしての 姿勢と心構え
recruitengineers
PRO
2
600
Featured
See All Featured
Facilitating Awesome Meetings
lara
55
6.5k
GraphQLとの向き合い方2022年版
quramy
49
14k
Building Adaptive Systems
keathley
43
2.7k
Practical Orchestrator
shlominoach
190
11k
YesSQL, Process and Tooling at Scale
rocio
173
14k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
61k
Become a Pro
speakerdeck
PRO
29
5.5k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Scaling GitHub
holman
462
140k
Intergalactic Javascript Robots from Outer Space
tanoku
272
27k
How to Ace a Technical Interview
jacobian
279
23k
Transcript
Better Docker Image+ Bonfire Backend #2 #yjbonfire @orisano
ྑ͍Docker ImageͱԿ͔
ݟͯΘ͔Γ͍͢ खݩͰߴʹbuild͞ΕΔ CI্Ͱߴʹbuild͞ΕΔ ߴʹdeploy͞ΕΔ
ࠓ͍ͨ͜͠ͱ
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
imageΛখ͘͢͞Δͷ docker push͢Δͱ͖ͷ docker pull͢Δͱ͖ͷ ͷߴԽʹͭͳ͕Δ
Ͳͷ༷ʹখ͘͢͞Δ͔ ͋ͱͰ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ίϚϯυͦͷͷΛ͘͢Δ
URLʹର͢ΔADDΛΘͳ͍
URLʹର͢ΔADD جຊతʹμϯϩʔυ͢Δ
͍
ΞΫηε͍ͯ͠Δઌͷ ίϯςϯπ͕ႈͳΒ wget + gzip + tarͰे
`RUN wget`ʹ͢Δ͜ͱͰ cache͕ޮ͘
ႈͰͳ͍Ϧιʔεʹ ґଘ͢ΔͷΛۃྗΊΔ
build context ͷసૹྔΛߟ͑Δ
ϞϊϨϙʹ͢Δͱ build contextେ͖͘ͳΓ͕ͪ
ϞϊϨϙͷ߹ .dockerignore Λࣗಈੜ͠Α͏
github.com/orisano/dignore • ࢦఆ͞ΕͨdirectoryҎ֎ͷdirectoryΛignore • ࢦఆ͞Εͨdirectoryʹdockerignore͕͋Ε ల։ • ͜Ε͚ͩʂ
buildͷલఔͰ buildʹඞཁͳ͍αʔϏε Ignore͠Α͏
buildkitΛ͍ͬͯΔͱ ࠩసૹͯ͘͠ΕͨΓ͢Δ
COPYΛҙࣝͨ͠ directoryߏ
COPYͷҾ͕ directoryͷ߹ ରͷdirectoryʹ தΛશ෦ίϐʔͯ͠͠·͏
ಛఆͷσΟϨΫτϦ͚ͩ ίϐʔͨ͘͠ͳ͍ ͱ͍͏͕͍͠
dockerignore Ͱআ֎͢Εྑ͘ͳ͍ʁ
͍͍͑
vendorͳͲΛཧͯ͠ build࣌ʹdownload͠ͳ͍ ߹ʹຊʹਏ͍
2ճҎ্ॏ͍directoryΛ COPYͨ͘͠ͳ͍
ͦ͏ͳΒͳ͍ͨΊͷ directoryߏʹ͓ͯ͜͠͏
ਓ͕ؒॻ͍ͨͷ͕ ೖ͍ͬͯΔdirectory ֎෦ʹґଘ͍ͯ͠Δͷ ࣗಈੜ͕ೖ͍ͬͯΔ directoryΛ͠Α͏
buildkitͩͱࠩసૹ
͍ͣΕʹͤΑ ͓͍ͯͨ͠ํ͕ cacheʹ༏͍͠
RUNͷதΛ͘͢Δ
GitHub͔Β࣮ߦϑΝΠϧΛ curl or wgetͰऔಘ͢Δ ͕͍
Ͳ͏ʹ͔͘Ͱ͖ͳ͍͔ curl -vvvΛ͏ͬͯΈͨ
GitHub ReleaseS3Ͱ͋Δ S3Accept-Ranges: bytes ͱ͍͏͜ͱ͕Θ͔ͬͨ
Accept-Ranges: bytes ͬͯͳΜͩΖ͏
RFC7233, Range Requests ൣғΛࢦఆͯ͠ downloadͰ͖Δ
ͭ·Γ ฒྻμϯϩʔυ͕Մೳ
github.com/orisano/rget
None
ڥʹΑΓ·͕͢ 5min -> 2min
DockerfileͷҎ֎ʹ ͘͢ΔΞϓϩʔν͋Δ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
cacheΛޮ͔ͤΔ
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
cacheͷΈΛཧղ͢Δ
RUN จࣈྻ͕มΘΒͳ͍ݶΓ جຊతʹcache͞ΕΔ
COPY, ADD͢ΔϑΝΠϧͷ ༰͕มΘͬͨ߹ Ҏ߱ͷRUNͷcache͕ഁغ
ͳͷͰ lockfileͳͲΛઌʹίϐʔͯ͠ install͚ͩͯ͠cacheͤ͞Δ
None
None
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
CI্Ͱbuild͢Δͱ͖ͳͲ cache͕ͳ͍͜ͱ͕ଟ͍
docker save & load or docker pull docker build —cache-from
Λ͏
image͕େ͖͍/layer͕ଟ͍ ߹ buildͷ΄͏͕ૣ͍͜ͱ͋Δ
image͕େ͖͍/layer͕ଟ͍ ߹ buildͷ΄͏͕ૣ͍͜ͱ͋Δ ܭଌܾͯ͠ΊΔ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ґଘͷͳ͍εςʔδΛ ฒྻͰ࣮ߦ͢Δ
buildkitΛ͍ͬͯͩ͘͞ʂ github.com/moby/buildkit export DOCKER_BUILDKIT=1
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ඞཁͳ͍εςʔδΛ build͠ͳ͍
—targetΛ͑Α͍ͷͰʁ
—target ࢦఆͨ͠εςʔδҎલΛ શ෦build͢Δ
multi stage buildΛ ౿ΈࠐΜ͍ͩํ͍ͯ͠Δͱ ૺ۰͕ͪ͠
multi stage buildͷ ౿ΈࠐΜ͍ͩํʹ͍ͭͯ
https://speakerdeck.com/orisano/multi-stage-builds-patterns-and-practice
͓ͦΒ͘buildkitͰͰ͖Δ͕ buildkit͕͑ͳ͍߹ʹ
github.com/orisano/targd • DockerfileͷASTΛऔಘ • ࢦఆ͞ΕͨεςʔδͷґଘεςʔδΛநग़ • ґଘεςʔδ͚ͩͷDockerfileΛग़ྗ
github.com/orisano/targd
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
multi stage buildΛ͏
multi stage buildͰ ͋Γ͕ͪͳٙ
࠷ऴతͳΠϝʔδ͕ খ͘͞ͳΔ͔Β ͦΕ·Ͱͷεςʔδ ࠷దԽ͠ͳͯ͘ྑ͍ʁ
None
ݸਓతʹNo
moby/issues/34715 —cache-formͩͱ multi stage buildͷ લஈͷεςʔδͷcache͕ ޮ͔ͳ͍
࠷ऴεςʔδ͔͠ pushͯ͠ͳ͍͔ΒͨΓલ
CI্Ͱͷbuild —cache-fromΛ͏ ࠷ऴεςʔδ͚ͩcache͢Δ ҙຯͳ͍
multi stage build࣌ʹ cacheΛޮ͔͍ͤͨ߹ લͷεςʔδ ໌ࣔతʹpush͢Δ͔͠ͳ͍
͜ΕΛखಈͰΔͱ cache-fromࠈʹͳΔ
github.com/orisano/castage • DockerfileͷASTΛऔಘ • εςʔδҰཡΛऔಘ • docker pullΛੜ • docker
buildΛcache-from͖Ͱੜ
݁ہpush͢ΔͷͰ push/pullͷίετ͕͔͔Δ ͯ͢ͷεςʔδΛฏʹ খ͖͘͢͞ (ݸਓͷݟղͰ͢)
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
RUNΛ·ͱΊΔ(?)
Ͳ͔ͬʔ;͍͊Δ 1ͭͷRUNʹ શ෦ॻ͘ͱྑ͍Β͍͠(?)
github.com/orisano/minid • DockerfileͷASTΛऔಘ • ࿈ଓ͢ΔRUN, COPY, ADDΛ࿈݁͢Δ • ݁ՌͷDockerfileΛग़ྗ
github.com/orisano/minid
αΠζ͕খ͘͞ͳͬͨ
ͰͳΜͰʁ
ϨΠϠʔͷΦʔόʔϔου͕ ݮΔ͔Βখ͘͞ͳΔʁ
͍͍͑
·ͣ Ͳ͏͍͏ܗͰอଘ͞ΕͯΔ͔ ΛΔ
moby/image/spec/v1.md
AUFS
http://docs.docker.jp/engine/userguide/storagedriver/aufs-driver.html
আ whiteoutϑΝΠϧͷՃ Ҡಈ opaqueϑΝΠϧͷՃ ʴ ҠಈઌͷϑΝΠϧࠩ
ҰͰ RUN,COPY,ADDΛލ͙ͱ imageʹͬͯ͠·͏
ػցతʹͰ ҰͭͷRUNʹ·ͱΊΔ͜ͱͰ ༨ܭͳͷΛݮͰ͖Δ
1ͭͷϨΠϠʔʹ ·ͱΊΔ͜ͱ ຊʹਖ਼͍͠ͷ͔
ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻
ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻ ܭଌܾͯ͠ΊΔ
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
ͳͥେ͖͍͔ΛΔ
͍͔ͳΔνϡʔχϯάͰ ܭଌͤͣʹ͍͚ͬͯͳ͍
docker history
docker history
ͲͷϨΠϠʔ͕ େ͖͍͔Θ͔Δ͚Ͳ ͳͥେ͖͍͔͔ΓͮΒ͍
ҙ֎ʹίϚϯυ͕ Ͳ͏͍͏ϑΝΠϧΛ࡞Δͷ͔ Βͳ͍
github.com/orisano/dlayer • docker saveͰಘΒΕͨtarΛղੳ • layerͱcmdͷରԠΛͱΔ • layerͷࠩΛϑΝΠϧαΠζ͕େ͖͍ॱʹද ࣔ
github.com/orisano/dlayer
࣮ࡍʹdlayerΛͬͯ golangެࣜalpineΠϝʔδͷ αΠζΛݮΒͯ͠Έͨ
docker-library/golang/pull/232
None
golang:1.11͔Βtoolchainͷ αΠζ͕Ͱ͔͘ͳͬͨ SSAͰͷ࠷దԽ͕૿͔͑ͨΒʁ
253MB
305MB
github.com/wagoodman/dive
࡞ͬͨπʔϧ • github.com/orisano/rget • github.com/orisano/targd • github.com/orisano/minid • github.com/orisano/dlayer •
ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢
࡞ͬͨπʔϧ • github.com/orisano/dignore • github.com/orisano/castage • ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢
·ͱΊ • ίϚϯυΛ͘Ͱ͖ͳ͍͔ߟ͑Δ • cacheΛཧղͯ͠༗ޮʹ׆͔͢ (CI or ϩʔΧϧ) • multi
stage buildΛ͏ (ॏ͍stageΛcache͢Δ) • layerΛখ͘͢͞ΔͨΊʹੳ͢Δ • buildkitΛ͏!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
rget͜΅Ε
github.com/orisano/rget
Ͱ ͜ΕͰຊʹ͍͍ΜͩΖ͏͔
rgetͷbinaryΛdownload ͢ΔͷͰෛ͚ͨؾ͕͢Δ()
shell͚ͩͰ ࣮ݱͰ͖ͳ͍ͩΖ͏͔
ϙʔλϒϧੑ͍Βͳ͍ͷͰ ओઓͷalpineʹߜΔ
alpine(busybox) xargs͕ೖ͍ͬͯΔʂʂʂ
xargsΛ͏͜ͱͰ ฒྻԽ ಉ࣌ଓ੍ݶ ͕࣮ݱͰ͖Δ
GitHub Release ؆୯ʹHEAD͕Ͱ͖ͳ͍ͷͰ GETͰbodyΛແࢹͯ͠ Content-LengthΛऔಘͰ͖Δ
seqͰ chunkͷrangeΛੜͰ͖Δ
wget206 Partial Content ͕͏·͘ѻ͑ͳ͍ͷͰμϝ
ํͳ͍ͷͰ curl -RͰdownload
࠷ޙʹcatͯ͠chunkΛ݁߹
shellͰ࣮ݱͰ͖Δʂ
͔͠͠ curlalpineඪ४Ͱͳ͍
apk add —no-cache curl ෛ͚ͨؾ͕͢Δ()
͔͠͠ wget206͕ॲཧͰ͖ͳ͍
ఘΊΒΕͳ͍ͷͰ busyboxͷwgetͷ ιʔείʔυΛಡΉ
busybox/networking/wget.c
busybox/networking/wget.c
-OͰࢦఆͨ͠ϑΝΠϧ͕ଘࡏ (ϑΝΠϧαΠζ͕1Ҏ্) -cΛࢦఆ͍ͯ͠Δ ্هͷ݅Λຬͨ͢ͱ͖ 206Λॲཧͯ͘͠ΕΔ
ͭ·Γ దͳ1byteͷϑΝΠϧʹ -cͰࢦఆ͢Δ͜ͱͰճආՄೳ
ઌ಄ͷ1byteΛऔΓআ͚Ε alpineඪ४ͷΈͰ࣮ݱՄೳ
tailͰՄೳ͕͍ͩ dd skip=1 iflag=skip_bytes ͕ߴ
alpineඪ४ͷΈͰ࣮ݱ
https://github.com/orisano/rget/blob/master/rget.sh