Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
better docker image
Search
orisano
October 09, 2018
Technology
22
30k
better docker image
orisano
October 09, 2018
Tweet
Share
More Decks by orisano
See All by orisano
OSS Performance Tuning Tips
orisano
8
5.8k
Docker-Compose & BuildKit
orisano
4
990
Container Build Talk
orisano
3
2.5k
dockerignore talk
orisano
2
7.1k
Better docker image+
orisano
6
6k
Socket.IO Introduction
orisano
0
3.1k
Profiling Go Application
orisano
11
7.7k
Multi-stage Builds Patterns & Practice
orisano
6
5.1k
the world of characters
orisano
8
1.4k
Other Decks in Technology
See All in Technology
CodePipelineのアクション統合から学ぶAWS CDKの抽象化技術 / codepipeline-actions-cdk-abstraction
gotok365
5
160
Creating Awesome Change in SmartNews
martin_lover
1
280
Devinで模索する AIファースト開発〜ゼロベースから始めるDevOpsの進化〜
potix2
PRO
7
3.4k
彩の国で始めよう。おっさんエンジニアから共有したい、当たり前のことを当たり前にする技術
otsuki
0
150
30代からでも遅くない! 内製開発の世界に飛び込み、最前線で戦うLLMアプリ開発エンジニアになろう
minorun365
PRO
6
560
Writing Ruby Scripts with TypeProf
mame
0
140
バクラクの認証基盤の成長と現在地 / bakuraku-authn-platform
convto
1
550
PagerDuty×ポストモーテムで築く障害対応文化/Building a culture of incident response with PagerDuty and postmortems
aeonpeople
1
120
ワールドカフェI /チューターを改良する / World Café I and Improving the Tutors
ks91
PRO
0
120
Linuxのパッケージ管理とアップデート基礎知識
go_nishimoto
0
240
SREからゼロイチプロダクト開発へ ー越境する打席の立ち方と期待への応え方ー / Product Engineering Night #8
itkq
2
800
AI AgentOps LT大会(2025/04/16) Algomatic伊藤発表資料
kosukeito
0
140
Featured
See All Featured
Reflections from 52 weeks, 52 projects
jeffersonlam
349
20k
[RailsConf 2023] Rails as a piece of cake
palkan
54
5.4k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.5k
Facilitating Awesome Meetings
lara
54
6.3k
Building Flexible Design Systems
yeseniaperezcruz
329
38k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.1k
Making the Leap to Tech Lead
cromwellryan
133
9.2k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
160
15k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
YesSQL, Process and Tooling at Scale
rocio
172
14k
Transcript
Better Docker Image Container Build Meetup #1 #container_build @orisano
ྑ͍Docker ImageͱԿ͔
αʔϏεʹؔΘΔਓΛ ͤʹ͢ΔΠϝʔδ
ݟͯΘ͔Γ͍͢ खݩͰߴʹbuild͞ΕΔ CI্Ͱߴʹbuild͞ΕΔ ߴʹdeploy͞ΕΔ
ࠓ͍ͨ͜͠ͱ
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
imageΛখ͘͢͞Δͷ docker push͢Δͱ͖ͷ docker pull͢Δͱ͖ͷ ͷߴԽʹͭͳ͕Δ
Ͳͷ༷ʹখ͘͢͞Δ͔ ͋ͱͰ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ίϚϯυͦͷͷΛ͘͢Δ
URLʹର͢ΔADDΛΘͳ͍ RUNͷதΛ͘͢Δ
URLʹର͢ΔADDΛΘͳ͍ RUNͷதΛ͘͢Δ
URLʹର͢ΔADD جຊతʹμϯϩʔυ͢Δ
͍
ΞΫηε͍ͯ͠Δઌͷ ίϯςϯπ͕ႈͳΒ wget + gzip + tarͰे
`RUN wget`ʹ͢Δ͜ͱͰ cache͕ޮ͘
ႈͰͳ͍Ϧιʔεʹ ґଘ͢ΔͷΛۃྗΊΔ
URLʹର͢ΔADDΛΘͳ͍ RUNͷதΛ͘͢Δ
GitHub͔Β࣮ߦϑΝΠϧΛ curl or wgetͰऔಘ͢Δ ͕͍
Ͳ͏ʹ͔͘Ͱ͖ͳ͍͔ curl -vvvΛ͏ͬͯΈͨ
GitHub ReleaseS3Ͱ͋Δ S3Accept-Ranges: bytes ͱ͍͏͜ͱ͕Θ͔ͬͨ
Accept-Ranges: bytes ͬͯͳΜͩΖ͏
RFC7233, Range Requests ൣғΛࢦఆͯ͠ downloadͰ͖Δ
ͭ·Γ ฒྻμϯϩʔυ͕Մೳ
github.com/orisano/rget
None
ڥʹΑΓ·͕͢ 5min -> 2min
DockerfileͷҎ֎ʹ ͘͢ΔΞϓϩʔν͋Δ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
cacheΛޮ͔ͤΔ
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
cacheͷΈΛཧղ͢Δ
RUN จࣈྻ͕มΘΒͳ͍ݶΓ جຊతʹcache͞ΕΔ
COPY, ADD͢ΔϑΝΠϧͷ ༰͕มΘͬͨ߹ Ҏ߱ͷRUNͷcache͕ഁغ
ͳͷͰ lockfileͳͲΛઌʹίϐʔͯ͠ install͚ͩͯ͠cacheͤ͞Δ
None
None
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
CI্Ͱbuild͢Δͱ͖ͳͲ cache͕ͳ͍͜ͱ͕ଟ͍
docker save & load or docker pull docker build —cache-from
Λ͏
image͕େ͖͍/layer͕ଟ͍ ߹ buildͷ΄͏͕ૣ͍͜ͱ͋Δ
image͕େ͖͍/layer͕ଟ͍ ߹ buildͷ΄͏͕ૣ͍͜ͱ͋Δ ܭଌܾͯ͠ΊΔ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ґଘͷͳ͍εςʔδΛ ฒྻͰ࣮ߦ͢Δ
buildkitΛ͍ͬͯͩ͘͞ʂ github.com/moby/buildkit export DOCKER_BUILDKIT=1
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ඞཁͳ͍εςʔδΛ build͠ͳ͍
—targetΛ͑Α͍ͷͰʁ
—target ࢦఆͨ͠εςʔδҎલΛ શ෦build͢Δ
multi stage buildΛ ౿ΈࠐΜ͍ͩํ͍ͯ͠Δͱ ૺ۰͕ͪ͠
͓ͦΒ͘buildkitͰͰ͖Δ͕ buildkit͕͑ͳ͍߹ʹ
github.com/orisano/targd • DockerfileͷASTΛऔಘ • ࢦఆ͞ΕͨεςʔδͷґଘεςʔδΛநग़ • ґଘεςʔδ͚ͩͷDockerfileΛग़ྗ
github.com/orisano/targd
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
multi stage buildΛ͏
multi stage buildͰ ͋Γ͕ͪͳٙ
࠷ऴతͳΠϝʔδ͕ খ͘͞ͳΔ͔Β ͦΕ·Ͱͷεςʔδ ࠷దԽ͠ͳͯ͘ྑ͍ʁ
None
ݸਓతʹNo
moby/issues/34715 —cache-formͩͱ multi stage buildͷ લஈͷεςʔδͷcache͕ ޮ͔ͳ͍
࠷ऴεςʔδ͔͠ pushͯ͠ͳ͍͔ΒͨΓલ
CI্Ͱͷbuild —cache-fromΛ͏ ࠷ऴεςʔδ͚ͩcache͢Δ ҙຯͳ͍
multi stage build࣌ʹ cacheΛޮ͔͍ͤͨ߹ લͷεςʔδ ໌ࣔతʹpush͢Δ͔͠ͳ͍
݁ہpush͢ΔͷͰ push/pullͷίετ͕͔͔Δ ͯ͢ͷεςʔδΛฏʹ খ͖͘͢͞ (ݸਓͷݟղͰ͢)
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
RUNΛ·ͱΊΔ(?)
Ͳ͔ͬʔ;͍͊Δ 1ͭͷRUNʹ શ෦ॻ͘ͱྑ͍Β͍͠(?)
github.com/orisano/minid • DockerfileͷASTΛऔಘ • ࿈ଓ͢ΔRUN, COPY, ADDΛ࿈݁͢Δ • ݁ՌͷDockerfileΛग़ྗ
github.com/orisano/minid
αΠζ͕খ͘͞ͳͬͨ
ͰͳΜͰʁ
ϨΠϠʔͷΦʔόʔϔου͕ ݮΔ͔Βখ͘͞ͳΔʁ
͍͍͑
·ͣ Ͳ͏͍͏ܗͰอଘ͞ΕͯΔ͔ ΛΔ
moby/image/spec/v1.md
AUFS
http://docs.docker.jp/engine/userguide/storagedriver/aufs-driver.html
আ whiteoutϑΝΠϧͷՃ Ҡಈ opaqueϑΝΠϧͷՃ ʴ ҠಈઌͷϑΝΠϧࠩ
ҰͰ RUN,COPY,ADDΛލ͙ͱ imageʹͬͯ͠·͏
ػցతʹͰ ҰͭͷRUNʹ·ͱΊΔ͜ͱͰ ༨ܭͳͷΛݮͰ͖Δ
1ͭͷϨΠϠʔʹ ·ͱΊΔ͜ͱ ຊʹਖ਼͍͠ͷ͔
ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻ Մಡੑͷ্
ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻ Մಡੑͷ্ ܭଌܾͯ͠ΊΔ
ͱ͍͑ ෳϨΠϠʔʹ ލ͕Βͳ͍ఔʹ ׂ͢Δͷ͕ྑͦ͞͏
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
ͳͥେ͖͍͔ΛΔ
͍͔ͳΔνϡʔχϯάͰ ܭଌͤͣʹ͍͚ͬͯͳ͍
docker history
docker history
ͲͷϨΠϠʔ͕ େ͖͍͔Θ͔Δ͚Ͳ ͳͥେ͖͍͔͔ΓͮΒ͍
ҙ֎ʹίϚϯυ͕ Ͳ͏͍͏ϑΝΠϧΛ࡞Δͷ͔ Βͳ͍
github.com/orisano/dlayer • docker saveͰಘΒΕͨtarΛղੳ • layerͱcmdͷରԠΛͱΔ • layerͷࠩΛϑΝΠϧαΠζ͕େ͖͍ॱʹද ࣔ
github.com/orisano/dlayer
࣮ࡍʹdlayerΛͬͯ golangެࣜalpineΠϝʔδͷ αΠζΛݮΒͯ͠Έͨ
docker-library/golang/pull/232
None
golang:1.11͔Βtoolchainͷ αΠζ͕Ͱ͔͘ͳͬͨ SSAͰͷ࠷దԽ͕૿͔͑ͨΒʁ
1.10: 253MB
1.11: 305MB
࡞ͬͨπʔϧ • github.com/orisano/rget • github.com/orisano/targd • github.com/orisano/minid • github.com/orisano/dlayer •
ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢
·ͱΊ • ίϚϯυΛ͘Ͱ͖ͳ͍͔ߟ͑Δ • cacheΛཧղͯ͠༗ޮʹ׆͔͢ (CI or ϩʔΧϧ) • multi
stage buildΛ͏ (ॏ͍stageΛcache͢Δ) • layerΛখ͘͢͞ΔͨΊʹੳ͢Δ • buildkitΛ͏!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
rget͜΅Ε
github.com/orisano/rget
Ͱ ͜ΕͰຊʹ͍͍ΜͩΖ͏͔
rgetͷbinaryΛdownload ͢ΔͷͰෛ͚ͨؾ͕͢Δ()
shell͚ͩͰ ࣮ݱͰ͖ͳ͍ͩΖ͏͔
ϙʔλϒϧੑ͍Βͳ͍ͷͰ ओઓͷalpineʹߜΔ
alpine(busybox) xargs͕ೖ͍ͬͯΔʂʂʂ
xargsΛ͏͜ͱͰ ฒྻԽ ಉ࣌ଓ੍ݶ ͕࣮ݱͰ͖Δ
GitHub Release ؆୯ʹHEAD͕Ͱ͖ͳ͍ͷͰ GETͰbodyΛແࢹͯ͠ Content-LengthΛऔಘͰ͖Δ
seqͰ chunkͷrangeΛੜͰ͖Δ
wget206 Partial Content ͕͏·͘ѻ͑ͳ͍ͷͰμϝ
ํͳ͍ͷͰ curl -RͰdownload
࠷ޙʹcatͯ͠chunkΛ݁߹
shellͰ࣮ݱͰ͖Δʂ
͔͠͠ curlalpineඪ४Ͱͳ͍
apk add —no-cache curl ෛ͚ͨؾ͕͢Δ()
͔͠͠ wget206͕ॲཧͰ͖ͳ͍
ఘΊΒΕͳ͍ͷͰ busyboxͷwgetͷ ιʔείʔυΛಡΉ
busybox/networking/wget.c
busybox/networking/wget.c
-OͰࢦఆͨ͠ϑΝΠϧ͕ଘࡏ (ϑΝΠϧαΠζ͕1Ҏ্) -cΛࢦఆ͍ͯ͠Δ ্هͷ݅Λຬͨ͢ͱ͖ 206Λॲཧͯ͘͠ΕΔ
ͭ·Γ దͳ1byteͷϑΝΠϧʹ -cͰࢦఆ͢Δ͜ͱͰճආՄೳ
ઌ಄ͷ1byteΛऔΓআ͚Ε alpineඪ४ͷΈͰ࣮ݱՄೳ
tailͰՄೳ͕͍ͩ dd skip=1 iflag=skip_bytes ͕ߴ
alpineඪ४ͷΈͰ࣮ݱ
https://github.com/orisano/rget/blob/master/rget.sh