Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Better docker image+
Search
orisano
December 05, 2018
Technology
6
5.7k
Better docker image+
orisano
December 05, 2018
Tweet
Share
More Decks by orisano
See All by orisano
OSS Performance Tuning Tips
orisano
8
5.5k
Docker-Compose & BuildKit
orisano
4
910
Container Build Talk
orisano
3
2.4k
dockerignore talk
orisano
2
6.9k
Socket.IO Introduction
orisano
0
2.9k
Profiling Go Application
orisano
11
7.5k
Multi-stage Builds Patterns & Practice
orisano
6
4.9k
better docker image
orisano
22
29k
the world of characters
orisano
8
1.3k
Other Decks in Technology
See All in Technology
FastConnect の冗長性
ocise
0
7.1k
エンジニア向け会社紹介資料
caddi_eng
15
250k
Evolving DevOps Teams and Flexible Organizational Culture
kakehashi
1
130
Envoy External AuthZとgRPC Extensionを利用した「頑張らない」Microservices認証認可基盤
andoshin11
0
140
夏休みの(最後の)宿題 for JuliaTokyo #12
antimon2
0
130
エンジニア採用ノウハウ0から継続的な採用に至るまで / Try recruiting engineers
ohageeq
0
130
音声AIエージェントの世界とRetell AI入門 / Introduction to the World of Voice AI Agents and Retell AI
rkaga
3
470
実践的なバグバウンティ入門
scgajge12
4
2.1k
RAGHack: Kickoff and RAG 101
pamelafox
0
200
Namespace, Now and Then
tagomoris
0
170
Binary Authorizationと友達になろう / Let's be friends with Binary Authorization
iselegant
2
130
実践ジオフェンス 効率的に開発するために
navitimejapan
PRO
2
130
Featured
See All Featured
Into the Great Unknown - MozCon
thekraken
28
1.4k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
33
1.7k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
248
20k
Pencils Down: Stop Designing & Start Developing
hursman
118
11k
A Modern Web Designer's Workflow
chriscoyier
690
190k
Adopting Sorbet at Scale
ufuk
72
8.9k
We Have a Design System, Now What?
morganepeng
48
7.1k
Large-scale JavaScript Application Architecture
addyosmani
508
110k
A designer walks into a library…
pauljervisheath
201
24k
Visualization
eitanlees
142
15k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
45
4.8k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
27
1.6k
Transcript
Better Docker Image+ Bonfire Backend #2 #yjbonfire @orisano
ྑ͍Docker ImageͱԿ͔
ݟͯΘ͔Γ͍͢ खݩͰߴʹbuild͞ΕΔ CI্Ͱߴʹbuild͞ΕΔ ߴʹdeploy͞ΕΔ
ࠓ͍ͨ͜͠ͱ
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
imageΛখ͘͢͞Δͷ docker push͢Δͱ͖ͷ docker pull͢Δͱ͖ͷ ͷߴԽʹͭͳ͕Δ
Ͳͷ༷ʹখ͘͢͞Δ͔ ͋ͱͰ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ίϚϯυͦͷͷΛ͘͢Δ
URLʹର͢ΔADDΛΘͳ͍
URLʹର͢ΔADD جຊతʹμϯϩʔυ͢Δ
͍
ΞΫηε͍ͯ͠Δઌͷ ίϯςϯπ͕ႈͳΒ wget + gzip + tarͰे
`RUN wget`ʹ͢Δ͜ͱͰ cache͕ޮ͘
ႈͰͳ͍Ϧιʔεʹ ґଘ͢ΔͷΛۃྗΊΔ
build context ͷసૹྔΛߟ͑Δ
ϞϊϨϙʹ͢Δͱ build contextେ͖͘ͳΓ͕ͪ
ϞϊϨϙͷ߹ .dockerignore Λࣗಈੜ͠Α͏
github.com/orisano/dignore • ࢦఆ͞ΕͨdirectoryҎ֎ͷdirectoryΛignore • ࢦఆ͞Εͨdirectoryʹdockerignore͕͋Ε ల։ • ͜Ε͚ͩʂ
buildͷલఔͰ buildʹඞཁͳ͍αʔϏε Ignore͠Α͏
buildkitΛ͍ͬͯΔͱ ࠩసૹͯ͘͠ΕͨΓ͢Δ
COPYΛҙࣝͨ͠ directoryߏ
COPYͷҾ͕ directoryͷ߹ ରͷdirectoryʹ தΛશ෦ίϐʔͯ͠͠·͏
ಛఆͷσΟϨΫτϦ͚ͩ ίϐʔͨ͘͠ͳ͍ ͱ͍͏͕͍͠
dockerignore Ͱআ֎͢Εྑ͘ͳ͍ʁ
͍͍͑
vendorͳͲΛཧͯ͠ build࣌ʹdownload͠ͳ͍ ߹ʹຊʹਏ͍
2ճҎ্ॏ͍directoryΛ COPYͨ͘͠ͳ͍
ͦ͏ͳΒͳ͍ͨΊͷ directoryߏʹ͓ͯ͜͠͏
ਓ͕ؒॻ͍ͨͷ͕ ೖ͍ͬͯΔdirectory ֎෦ʹґଘ͍ͯ͠Δͷ ࣗಈੜ͕ೖ͍ͬͯΔ directoryΛ͠Α͏
buildkitͩͱࠩసૹ
͍ͣΕʹͤΑ ͓͍ͯͨ͠ํ͕ cacheʹ༏͍͠
RUNͷதΛ͘͢Δ
GitHub͔Β࣮ߦϑΝΠϧΛ curl or wgetͰऔಘ͢Δ ͕͍
Ͳ͏ʹ͔͘Ͱ͖ͳ͍͔ curl -vvvΛ͏ͬͯΈͨ
GitHub ReleaseS3Ͱ͋Δ S3Accept-Ranges: bytes ͱ͍͏͜ͱ͕Θ͔ͬͨ
Accept-Ranges: bytes ͬͯͳΜͩΖ͏
RFC7233, Range Requests ൣғΛࢦఆͯ͠ downloadͰ͖Δ
ͭ·Γ ฒྻμϯϩʔυ͕Մೳ
github.com/orisano/rget
None
ڥʹΑΓ·͕͢ 5min -> 2min
DockerfileͷҎ֎ʹ ͘͢ΔΞϓϩʔν͋Δ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
cacheΛޮ͔ͤΔ
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
cacheͷΈΛཧղ͢Δ
RUN จࣈྻ͕มΘΒͳ͍ݶΓ جຊతʹcache͞ΕΔ
COPY, ADD͢ΔϑΝΠϧͷ ༰͕มΘͬͨ߹ Ҏ߱ͷRUNͷcache͕ഁغ
ͳͷͰ lockfileͳͲΛઌʹίϐʔͯ͠ install͚ͩͯ͠cacheͤ͞Δ
None
None
docker build͢ΔϚγϯ͕ ಉҰͷ߹ ಉҰͰͳ͍߹
CI্Ͱbuild͢Δͱ͖ͳͲ cache͕ͳ͍͜ͱ͕ଟ͍
docker save & load or docker pull docker build —cache-from
Λ͏
image͕େ͖͍/layer͕ଟ͍ ߹ buildͷ΄͏͕ૣ͍͜ͱ͋Δ
image͕େ͖͍/layer͕ଟ͍ ߹ buildͷ΄͏͕ૣ͍͜ͱ͋Δ ܭଌܾͯ͠ΊΔ
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ґଘͷͳ͍εςʔδΛ ฒྻͰ࣮ߦ͢Δ
buildkitΛ͍ͬͯͩ͘͞ʂ github.com/moby/buildkit export DOCKER_BUILDKIT=1
ͲͷΑ͏ʹ͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷͷΛ͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
ඞཁͳ͍εςʔδΛ build͠ͳ͍
—targetΛ͑Α͍ͷͰʁ
—target ࢦఆͨ͠εςʔδҎલΛ શ෦build͢Δ
multi stage buildΛ ౿ΈࠐΜ͍ͩํ͍ͯ͠Δͱ ૺ۰͕ͪ͠
multi stage buildͷ ౿ΈࠐΜ͍ͩํʹ͍ͭͯ
https://speakerdeck.com/orisano/multi-stage-builds-patterns-and-practice
͓ͦΒ͘buildkitͰͰ͖Δ͕ buildkit͕͑ͳ͍߹ʹ
github.com/orisano/targd • DockerfileͷASTΛऔಘ • ࢦఆ͞ΕͨεςʔδͷґଘεςʔδΛநग़ • ґଘεςʔδ͚ͩͷDockerfileΛग़ྗ
github.com/orisano/targd
ͲͷΑ͏ʹ͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
multi stage buildΛ͏
multi stage buildͰ ͋Γ͕ͪͳٙ
࠷ऴతͳΠϝʔδ͕ খ͘͞ͳΔ͔Β ͦΕ·Ͱͷεςʔδ ࠷దԽ͠ͳͯ͘ྑ͍ʁ
None
ݸਓతʹNo
moby/issues/34715 —cache-formͩͱ multi stage buildͷ લஈͷεςʔδͷcache͕ ޮ͔ͳ͍
࠷ऴεςʔδ͔͠ pushͯ͠ͳ͍͔ΒͨΓલ
CI্Ͱͷbuild —cache-fromΛ͏ ࠷ऴεςʔδ͚ͩcache͢Δ ҙຯͳ͍
multi stage build࣌ʹ cacheΛޮ͔͍ͤͨ߹ લͷεςʔδ ໌ࣔతʹpush͢Δ͔͠ͳ͍
͜ΕΛखಈͰΔͱ cache-fromࠈʹͳΔ
github.com/orisano/castage • DockerfileͷASTΛऔಘ • εςʔδҰཡΛऔಘ • docker pullΛੜ • docker
buildΛcache-from͖Ͱੜ
݁ہpush͢ΔͷͰ push/pullͷίετ͕͔͔Δ ͯ͢ͷεςʔδΛฏʹ খ͖͘͢͞ (ݸਓͷݟղͰ͢)
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
RUNΛ·ͱΊΔ(?)
Ͳ͔ͬʔ;͍͊Δ 1ͭͷRUNʹ શ෦ॻ͘ͱྑ͍Β͍͠(?)
github.com/orisano/minid • DockerfileͷASTΛऔಘ • ࿈ଓ͢ΔRUN, COPY, ADDΛ࿈݁͢Δ • ݁ՌͷDockerfileΛग़ྗ
github.com/orisano/minid
αΠζ͕খ͘͞ͳͬͨ
ͰͳΜͰʁ
ϨΠϠʔͷΦʔόʔϔου͕ ݮΔ͔Βখ͘͞ͳΔʁ
͍͍͑
·ͣ Ͳ͏͍͏ܗͰอଘ͞ΕͯΔ͔ ΛΔ
moby/image/spec/v1.md
AUFS
http://docs.docker.jp/engine/userguide/storagedriver/aufs-driver.html
আ whiteoutϑΝΠϧͷՃ Ҡಈ opaqueϑΝΠϧͷՃ ʴ ҠಈઌͷϑΝΠϧࠩ
ҰͰ RUN,COPY,ADDΛލ͙ͱ imageʹͬͯ͠·͏
ػցతʹͰ ҰͭͷRUNʹ·ͱΊΔ͜ͱͰ ༨ܭͳͷΛݮͰ͖Δ
1ͭͷϨΠϠʔʹ ·ͱΊΔ͜ͱ ຊʹਖ਼͍͠ͷ͔
ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻
ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻ ܭଌܾͯ͠ΊΔ
ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔ΛΔ
ͳͥେ͖͍͔ΛΔ
͍͔ͳΔνϡʔχϯάͰ ܭଌͤͣʹ͍͚ͬͯͳ͍
docker history
docker history
ͲͷϨΠϠʔ͕ େ͖͍͔Θ͔Δ͚Ͳ ͳͥେ͖͍͔͔ΓͮΒ͍
ҙ֎ʹίϚϯυ͕ Ͳ͏͍͏ϑΝΠϧΛ࡞Δͷ͔ Βͳ͍
github.com/orisano/dlayer • docker saveͰಘΒΕͨtarΛղੳ • layerͱcmdͷରԠΛͱΔ • layerͷࠩΛϑΝΠϧαΠζ͕େ͖͍ॱʹද ࣔ
github.com/orisano/dlayer
࣮ࡍʹdlayerΛͬͯ golangެࣜalpineΠϝʔδͷ αΠζΛݮΒͯ͠Έͨ
docker-library/golang/pull/232
None
golang:1.11͔Βtoolchainͷ αΠζ͕Ͱ͔͘ͳͬͨ SSAͰͷ࠷దԽ͕૿͔͑ͨΒʁ
253MB
305MB
github.com/wagoodman/dive
࡞ͬͨπʔϧ • github.com/orisano/rget • github.com/orisano/targd • github.com/orisano/minid • github.com/orisano/dlayer •
ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢
࡞ͬͨπʔϧ • github.com/orisano/dignore • github.com/orisano/castage • ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢
·ͱΊ • ίϚϯυΛ͘Ͱ͖ͳ͍͔ߟ͑Δ • cacheΛཧղͯ͠༗ޮʹ׆͔͢ (CI or ϩʔΧϧ) • multi
stage buildΛ͏ (ॏ͍stageΛcache͢Δ) • layerΛখ͘͢͞ΔͨΊʹੳ͢Δ • buildkitΛ͏!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
rget͜΅Ε
github.com/orisano/rget
Ͱ ͜ΕͰຊʹ͍͍ΜͩΖ͏͔
rgetͷbinaryΛdownload ͢ΔͷͰෛ͚ͨؾ͕͢Δ()
shell͚ͩͰ ࣮ݱͰ͖ͳ͍ͩΖ͏͔
ϙʔλϒϧੑ͍Βͳ͍ͷͰ ओઓͷalpineʹߜΔ
alpine(busybox) xargs͕ೖ͍ͬͯΔʂʂʂ
xargsΛ͏͜ͱͰ ฒྻԽ ಉ࣌ଓ੍ݶ ͕࣮ݱͰ͖Δ
GitHub Release ؆୯ʹHEAD͕Ͱ͖ͳ͍ͷͰ GETͰbodyΛແࢹͯ͠ Content-LengthΛऔಘͰ͖Δ
seqͰ chunkͷrangeΛੜͰ͖Δ
wget206 Partial Content ͕͏·͘ѻ͑ͳ͍ͷͰμϝ
ํͳ͍ͷͰ curl -RͰdownload
࠷ޙʹcatͯ͠chunkΛ݁߹
shellͰ࣮ݱͰ͖Δʂ
͔͠͠ curlalpineඪ४Ͱͳ͍
apk add —no-cache curl ෛ͚ͨؾ͕͢Δ()
͔͠͠ wget206͕ॲཧͰ͖ͳ͍
ఘΊΒΕͳ͍ͷͰ busyboxͷwgetͷ ιʔείʔυΛಡΉ
busybox/networking/wget.c
busybox/networking/wget.c
-OͰࢦఆͨ͠ϑΝΠϧ͕ଘࡏ (ϑΝΠϧαΠζ͕1Ҏ্) -cΛࢦఆ͍ͯ͠Δ ্هͷ݅Λຬͨ͢ͱ͖ 206Λॲཧͯ͘͠ΕΔ
ͭ·Γ దͳ1byteͷϑΝΠϧʹ -cͰࢦఆ͢Δ͜ͱͰճආՄೳ
ઌ಄ͷ1byteΛऔΓআ͚Ε alpineඪ४ͷΈͰ࣮ݱՄೳ
tailͰՄೳ͕͍ͩ dd skip=1 iflag=skip_bytes ͕ߴ
alpineඪ४ͷΈͰ࣮ݱ
https://github.com/orisano/rget/blob/master/rget.sh