Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Container Build Talk
Search
Sponsored
·
Ship Features Fearlessly
Turn features on and off without deploys. Used by thousands of Ruby developers.
→
orisano
July 25, 2019
Programming
2.6k
3
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Container Build Talk
orisano
July 25, 2019
Transcript
134 slides
Container Buildͷ @orisano
Agenda • docker buildͷ • CI ʹ͓͚Δdocker build • docker
buildʹඞཁͩͬͨͷ
Agenda • docker buildͷ • CI ʹ͓͚Δdocker build • docker
buildʹඞཁͩͬͨͷ
EPDLFSDMJ EPDLFSE HTTP
EPDLFSDMJ EPDLFSE HTTP Mac LinuxKit
EPDLFSDMJ EPDLFSE HTTP Mac LinuxKit $ docker build [build context]
EPDLFSDMJ EPDLFSE HTTP Mac LinuxKit $ docker build [build context]
directoryҎԼΛ.dockerignore Λߟྀͯ͠tarʹ͢Δ
EPDLFSDMJ EPDLFSE HTTP Mac LinuxKit $ docker build [build context]
directoryҎԼΛ.dockerignore Λߟྀͯ͠tarʹ͢Δ tar
Dockerfile
Dockerfile ෳͷεςʔδΛ࣋ͭ εςʔδ ≒ Πϝʔδ
Stage FROM image [AS stage] RUN apk add ca-certificates COPY
package.json . COPY package-lock.json . ENV NODE_ENV=production RUN npm ci COPY . . RUN npm run build
Dockerfile Stage 1 Stage 2 Stage 3
ҰͷbuildͰ࡞ΕΔͷ 1Πϝʔδ͚ͩ
جຊతʹ ࠷ޙͷεςʔδ͕ग़ྗ͞ΕΔ
Dockerfile Stage 1 Stage 2 Stage 3
—targetͰࢦఆ͢Δͱ ग़ྗΠϝʔδΛܾΊΒΕΔ
Dockerfile Stage 1 Stage 2 Stage 3 —target
Πϝʔδ ϨΠϠʔͷੵΈॏͶͰ ࡞ΒΕΔ
ϨΠϠʔ͕࡞ΒΕΔͷ COPY/ADD, RUN ͷλΠϛϯά
ϕʔεΠϝʔδ
ϕʔεΠϝʔδ $01:"%%
ϕʔεΠϝʔδ $01:"%% ϕʔεΠϝʔδ 36/
ϕʔεΠϝʔδ $01:"%% ϕʔεΠϝʔδ 36/ $01:"%%
ϕʔεΠϝʔδ $01:"%% ϕʔεΠϝʔδ 36/ $01:"%% 36/
36/ tar app/testdata app/.wh.large_file app/a.txt app/b.txt
ϨΠϠʔ ࠩͷϑΝΠϧΛ࣋ͭtar
আ.wh.͔Β࢝·Δ whiteoutϑΝΠϧΛ࡞Δ͚ͩ
ίϚϯυΛލ͍Ͱআͯ͠ Γଓ͚Δ
ϨΠϠʔ ͷࢀরΛ͚ͩΛ࣋ͭ
୯ҰΠϝʔδͷ߹
ϕʔεΠϝʔδ $01:"%% ϕʔεΠϝʔδ 36/ $01:"%% 36/
ෳΠϝʔδͷ߹
ϕʔεΠϝʔδ $01:"%% ϕʔεΠϝʔδ 36/ $01:"%% 36/ $01:"%% 36/ $01:"%%
Build Cache ͷ ϝΧχζϜ
ϕʔεΠϝʔδ $01:"%% ϕʔεΠϝʔδ 36/ $01:"%% 36/ $01:"%% 36/ $01:"%% ίϐʔͨ͠༰Ͱ
ίϚϯυ͕มΘΔ
ϕʔεΠϝʔδ $01:"%% ϕʔεΠϝʔδ 36/ $01:"%% 36/ $01:"%% 36/ $01:"%% ίϐʔͨ͠༰Ͱ
ίϚϯυ͕มΘΔ ڞ௨ͷΛ࣋ͭಉ͡ίϚϯυͷΠϝʔδͰ ࠷৽ͷͷΛΩϟογϡͱͯ͠༻͍Δ
ϕʔεΠϝʔδ $01:"%% ϕʔεΠϝʔδ 36/ $01:"%% 36/ $01:"%% 36/ $01:"%% ίϐʔͨ͠༰Ͱ
ίϚϯυ͕มΘΔ ڞ௨ͷΛ࣋ͭಉ͡ίϚϯυͷΠϝʔδͰ ࠷৽ͷͷΛΩϟογϡͱͯ͠༻͍Δ Dockerd
άϥϑͱͯ͠ ঢ়ଶΛ͍࣋ͬͯΔͷ dockerd
Agenda • docker buildͷ • CI ʹ͓͚Δdocker build • docker
buildʹඞཁͩͬͨͷ
CI্Ͱdocker build cache͕ޮ͔ͳ͍
ͳͥͳΒ dockerd͕ͣͬͱੜ͖͍ͯΔ Θ͚Ͱͳ͍͔Β
ڞ௨ͷΛ࣋ͭ ΠϝʔδͳͲ͍ͳ͍
ղܾࡦ
docker pullͯ͘͠Δ
લʹbuildͨ͠ ΠϝʔδΛpull͓͚ͯ͠ cache͕ޮ͘?
͍͍͑
֎෦͔Β͖࣋ͬͯͨΠϝʔδ —cache-from͕ͳ͍ͱର֎
͜ΕͰղܾʂ
ͦΜͳ͕࣌͋Γ·ͨ͠
ࠓ େmulti stage build࣌
աڈͷৗࣝ (෦తʹ)௨༻͠ͳ͍
εςʔδ͝ͱʹ Πϝʔδ͕࡞ΒΕΔ
Dockerfile Stage 1 Stage 2 Stage 3
ҰͷbuildͰ࡞ΕΔͷ 1Πϝʔδ͚ͩ
Dockerfile Stage 1 Stage 2 Stage 3 —target
cacheͷ࠷খ୯Ґ͕ Πϝʔδ
ಛఆͷεςʔδΛ cacheΛޮ͔ͤͯbuild͢Δ ͨΊʹ Ҏલͷεςʔδ͕શͯඞཁ
શ෦pull શ෦cache-from
docker pull application:build-base-cache || true docker build -t application:build-base-cache --target=build-base
--cache- from=application:build-base-cache . docker pull application:base-cache || true docker build -t application:base-cache --target=base --cache-from=application:build-base- cache,application:base-cache . docker pull application:app-build-cache || true docker build -t application:app-build-cache --target=app-build --cache- from=application:build-base-cache,application:base-cache,application:app-build-cache . docker pull application:app-base-cache || true docker build -t application:app-base-cache --target=app-base --cache- from=application:build-base-cache,application:base-cache,application:app-build- cache,application:app-base-cache . docker pull application:application-cache || true docker build -t application:application-cache --target=application --cache- from=application:build-base-cache,application:base-cache,application:app-build- cache,application:app-base-cache,application:application-cache .
߇͑Ίʹ͍ͬͯ ࠈ
Agenda • docker buildͷ • CI ʹ͓͚Δdocker build • docker
buildʹඞཁͩͬͨͷ
ඞཁͩͬͨͷͳʹ͔
docker buildҎ֎ͷ πʔϧΛݟͯΈΔ
uber/makisuͷ
makisu Uber͕࡞ͬͨbuilder
https://eng.uber.com/makisu/
2015͘Β͍͔Β DockerҠߦΛ࢝ΊͨUber
Apache MesosͱK8S
400αʔϏε͘Β͍
docker buildͷprocess ࣗಈԽ, ඪ४Խ
ػີใͷऔѻʹ·ͣࠔͬͨ
docker-squashͰղܾʂ
͔͠͠ build͕࣌ؒ2ഒʹ
͍ʹͳΒͳ͍ͷͰ dockerΛfork͢Δ͜ͱʹ
build࣌ʹ volumeΛmountͰ͖ΔΑ͏ʹ
େຬ
2017 3000αʔϏε·Ͱ
buildʹ2͔͔࣌ؒΓ 10GBΛ͑ΔΠϝʔδ
storage, ଳҬ, ੜ࢈ੑʹ μϝʔδ
εέʔϧ͢Δ ࣍ੈͷϏϧυʹ͍ͭͯߟ͑Δ
ϙʔλϒϧ ࢄΩϟογϡ αΠζ࠷దԽ
ϙʔλϒϧ ࢄΩϟογϡ αΠζ࠷దԽ
2017ʹ ࣗಈԽ͞Ε, εέʔϥϒϧͰ ޮతͳΠϯϑϥ͕͋ͬͨ
Docker build ͦͷ্Ͱಈ͔͔ͨͬͨ͠
docker build copy-on-writeͰࠩΛ ͍Ζ͍Ζ͍ͬͯΔͷͰ ڧ͍ݖݶ͕ඞཁ
ڧ͍ΫϥελͰ ηΩϡϦςΟ ճආ͍ͨ͠
ϙʔλϒϧ ࢄΩϟογϡ αΠζ࠷దԽ
LayerCacheͰ લճͷLayerΛ͍ճͤΔͱ build࣌ؒΛ͘Ͱ͖Δ
Dockerͷcache ϒϥϯνؒผαʔϏεͩͱ ޮ͔ͳ͍
build machineͷׂͰ cache hitΛ্͕ͤͨ͞ ෳࡶ্͕͕ͬͯ͠·ͬͨ
ϙʔλϒϧ ࢄΩϟογϡ αΠζ࠷దԽ
খ͍͞Πϝʔδ ࠷ߴ
storage network decompress ʹޮ͘
multi stage build ྑ͍ղܾࡦ
͔͠͠ Dockerfile͕ෳࡶʹͳΔ
ͦͷ΄͔ʹ layerΛ·͍ͨͰআ͍ͯ͠Δ ΠϝʔδͳͲ͋ͬͨ
makisu 3ͭͷΛղܾ͢Δ
ϙʔλϒϧ ࢄΩϟογϡ αΠζ࠷దԽ
ϙʔλϒϧ ࢄΩϟογϡ αΠζ࠷దԽ
makisu copy on writeΛΘͣ In Memory FSͰࠩΛऔΔ ৄ͘͠ޙड़
ऄ
Dockerlayerͷѹॖʹ GoͷgzipΛ͍ͬͯΔ͕ pgzipͷ΄͏͕͍ͷͰ makisuͦͬͪ
ϙʔλϒϧ ࢄΩϟογϡ αΠζ࠷దԽ
Redis(or FS or HTTP)ʹ digestͱRegistryͷ ώϞ͕͋Γ Cache͕ղܾ͞ΕΔ
ϙʔλϒϧ ࢄΩϟογϡ αΠζ࠷దԽ
ಠࣗͷDockerfile parserΛ ͬͯ ໌ࣔతʹϨΠϠʔΛ࡞Δ ػߏ͕͋Δ
Stage FROM image [AS stage] RUN apk add ca-certificates COPY
package.json . COPY package-lock.json . ENV NODE_ENV=production RUN npm ci #!COMMIT COPY . . RUN npm run build
Stage FROM image [AS stage] RUN apk add ca-certificates COPY
package.json . COPY package-lock.json . ENV NODE_ENV=production RUN npm ci #!COMMIT COPY . . RUN npm run build
͜ͷػߏ͕͋ΕػີใΛ COPYͨ͋͠ͱʹআͯ͠ COMMITΛ͢Δͱ͍͏ࣄ͕ Dockerfile͚ͩͰ࣮ݱՄೳ
kaniko
kaniko Google͕࡞ͬͨbuilder
ίϯςφ্Ͱಈ͘
͜Ε·Ͱίϯςφ্Ͱͷ buildͬͯͲ͏ͯͨ͠ͷʁ
Privileged Container EPDLFSDMJ EPDLFSE HTTP
Container EPDLFSDMJ EPDLFSE HTTP Host
Container EPDLFSDMJ EPDLFSE HTTP Remote
kaniko
Container LBOJLP SPPUGT
Container LBOJLP SPPUGT 4($4 State &$3($3 Layer Cache
Container LBOJLP SPPUGT 4($4 State &$3($3 Layer Cache exec
Container LBOJLP SPPUGT 4($4 State &$3($3 Layer Cache exec snapshot
Container LBOJLP SPPUGT 4($4 State &$3($3 Layer Cache exec snapshot
Container LBOJLP 4($4 State &$3($3 SPPUGT &$3($3 Layer Cache Image
exec snapshot Registry
ঢ়ଶΛ ίϯςφ෦ʹ࣋ͨͳ͍
Image୯ҐͰͷ Pull/CacheͰͳ͘ Layer୯Ґ
BuildͱCacheͷߋ৽͕ ಉ࣌ʹߦΘΕΔ
Multi Stage Build ͰCache͕ޮ͘
ImageઈରPush͢Δ
ίϯςφΛͬͨ Build/Pushʹద͍ͯ͠Δ
kanikoʹ͍ͭͯৄ͘͠ kaniko ͕ԿΛ͍ͯ͠Δ͔, Կ͕Ͱ͖Δ͔ https://orisano.hatenablog.com/entry/2019/05/20/120032
makisuͱkaniko جຊతʹಉ͡࡞Γ
kaniko ͷ΄͏͕ Redisͱ͔͍Βͳ͍ͷͰ ͍͍͢
kaniko͓͢͢ΊͰ͢
Dockerfileͷ͜ͱ Docker buildͷ͜ͱͰ ࠔͬͨΒؾܰʹ twitter: @orisano
More Decks by orisano
See All by orisano
OSS Performance Tuning Tips
orisano
8
6.3k
Docker-Compose & BuildKit
orisano
4
1.1k
dockerignore talk
orisano
2
7.4k
Better docker image+
orisano
6
6.8k
Socket.IO Introduction
orisano
0
3.4k
Profiling Go Application
orisano
11
8.2k
Multi-stage Builds Patterns & Practice
orisano
6
5.4k
better docker image
orisano
22
31k
the world of characters
orisano
8
1.5k
Other Decks in Programming
See All in Programming
AppFunctionsの内部実装を紐解く
hiroaki404
0
100
私なりのハーネスエンジニアリング ~独自の言語処理系の開発を通して~
seosoft
0
230
flutter_test のその先へ — Flutter アプリの LLM 機能に「評価」という品質保証を
kosukesaigusa
0
210
Proxmox x Talos Linux x Terraform で実現するなんかいい感じの Kubernetes クラスタ / A Sleek Kubernetes Cluster Powered by Proxmox, Talos Linux, and Terraform
nnaka2992
0
280
動画プレイヤーはなぜカクつくのか 〜Flutterのレンダリングとネイティブの境界を解剖する〜
naruogram
0
340
AIがコードを書いてくれる時代、我々は何を確かめるのか
yosuke_furukawa
PRO
13
6.1k
LLM 101: Exploring LLMs, Starting with Tokens.
dalinaum
0
120
小さなアプリを作って公開するまで
onozaty
0
210
Kotlin Fest 2026: Kotlin コンパイラに Contribute してみよう! - Kotlin/Wasm で深い再帰を書けるようにした話
ternbusty
1
400
ローカルLLMの基本と近況 2611 / Local LLM basics and recent situation 2611
kishida
3
630
SODA - FACT BOOK(JP)
sodainc
1
40k
祈るテストから、探索するテストへ 〜Lincheckではじめる並行処理テスト〜
yyh_gl
0
400
Featured
See All Featured
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
2k
30 Presentation Tips
portentint
PRO
1
480
A brief & incomplete history of UX Design for the World Wide Web: 1989–2019
jct
2
570
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
38
3.1k
How To Speak Unicorn (iThemes Webinar)
marktimemedia
1
640
How Software Deployment tools have changed in the past 20 years
geshan
3
35k
AI Search: Where Are We & What Can We Do About It?
aleyda
0
8.3k
How to build an LLM SEO readiness audit: a practical framework
nmsamuel
2
990
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
508
140k
Applied NLP in the Age of Generative AI
inesmontani
PRO
4
2.6k
ラッコキーワード サービス紹介資料
rakko
5
6M
Designing for Performance
lara
612
70k