Upgrade to Pro — share decks privately, control downloads, hide ads and more …

(DockerCon 23) Container Images: Interactive De...

(DockerCon 23) Container Images: Interactive Deep Dive

Yves Brissaud

October 04, 2023
Tweet

More Decks by Yves Brissaud

Other Decks in Programming

Transcript

  1. → Images → Images, tags, pull → Images, tags, images

    internal Why To Care About Images? 00 Intro - Docker Hub Registry - Publishers (DVP, DSOS, …) Analytics - Docker scout
  2. What this talk is (not) about 00 Intro ✓ Build

    & Inspect multi-platform image ✓ Push & registry storage ✓ Pull & tags ✓ Update and new tags ✓ Beyond “images” ⛌ Image specifications by the book https://github.com/opencontainers/image-spec https://www.pexels.com/photo/close-up-shot-of-keys-on-a-red-surface-2882687/
  3. 01 Build Let’s build an image • Using a base

    image • For multiple architectures • Including SSC materials • Published on different tags
  4. 02 Inspect Extract and Inspect • Extract the image to

    a local directory • Explore starting with index.json
  5. 02 Inspect Image Index application/vnd.oci.image.index.v1+json linux/amd64 Image Manifest application/vnd.oci.image.m anifest.v1+json

    linux/arm64 Image Manifest application/vnd.oci.image.m anifest.v1+json attestation-manifest application/vnd.oci.image.m anifest.v1+json a>esta?on-manifest applica?on/vnd.oci.image.m anifest.v1+json
  6. Image Index (application/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … attestation-manifest attestation-manifest 02 Inspect
  7. Image Index (application/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … attestation-manifest attestation-manifest 02 Inspect “Image”
  8. Image Index (applica?on/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … attestation-manifest attestation-manifest 02 Inspect Multi platform image
  9. Image Index (application/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … a9esta<on-manifest Config Blob Layer application/vnd.in-toto+json Layer application/vnd.in-toto+json attestation-manifest Config Blob Layer applica?on/vnd.in-toto+json Layer application/vnd.in-toto+json 02 Inspect
  10. Why to push to a registry? 03 Push Why not

    just to share archives? ✔ Deduplication ✔ “Metadata” (tags) ✔ Versions https://www.pexels.com/photo/question-mark-on-crumpled-paper-5428826/
  11. 03 Push v2 blobs sha256 d6 d64d84c… a8 a85ae31... fd

    fd03efd... ... Registry View - Blobs
  12. 03 Push Registry View - Tags v2 repositories <repo name>

    _manifests tags latest current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ...
  13. 03 Push Registry View - Tags v2 repositories <repo name>

    _manifests tags latest current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ...
  14. 03 Push my/image:latest v2 repositories <repo name> _manifests tags latest

    current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ...
  15. 03 Push my/image:latest@sha256:… v2 repositories <repo name> _manifests tags latest

    current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ...
  16. 03 Push Registry View v2 repositories <repo name> _manifests tags

    latest current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ... blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  17. 03 Push Registry View v2 repositories <repo name> _manifests tags

    latest current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ... blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  18. Pull linux/amd64 version of latest 04 Pull 1. Convert tag

    to digest 2. Select the image for the right platform 3. Download config and layer blobs
  19. 04 Pull HEAD /v2/dc23/manifests/latest Convert tag to digest HTTP/1.1 200

    OK content-type: application/vnd.oci.image.index.v1+json docker-content-digest: sha256:5d0cbb38e39004b97dad3beb62fdde74e51f2f dcec80f547baa7ee5ed556cb4c docker-distribution-api-version: registry/2.0
  20. 04 Pull Convert tag to digest v2 repositories <repo name>

    _manifests tags latest current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ... blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  21. 04 Pull Convert tag to digest v2 repositories <repo name>

    _manifests tags latest current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ... blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  22. 04 Pull GET /v2/dc23/manifests/sha256:….. Find the right manifest { "mediaType":

    "application/vnd.oci.image.index.v1+json", "schemaVersion": 2, "manifests": [ { "mediaType": "application/vnd.oci.image.manifest.v1+json", "digest": "sha256:d64d84c3e5d2aa34243921261687bf482631dbd1d34c4890e94a13f392d9 bfa1", "size": 1812, "platform": { "architecture": "amd64", "os": "linux" } },
  23. Image Index (application/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … a9esta<on-manifest Config Blob Layer Layer attestation-manifest Config Blob Layer Layer 04 Pull
  24. 04 Pull GET /v2/dc23/manifests/sha256:d64… Find the right manifest { "mediaType":

    "application/vnd.oci.image.manifest.v1+json", "schemaVersion": 2, "config": { "mediaType": "application/vnd.oci.image.config.v1+json", "digest": "sha256:e999e4251aa2c2f7c0d8846883ea6e6dace050f5c07da7103137f4972df4 e97f", "size": 6896 }, "layers": [ { "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip", "digest": "sha256:9398808236ffac29e60c04ec906d8d409af7fa19dc57d8c65ad167e9c496 7006", "size": 3378609 },
  25. Image Index (application/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … a9esta<on-manifest Config Blob Layer Layer attestation-manifest Config Blob Layer Layer 04 Pull
  26. 04 Pull GET /v2/dc23/blobs/sha256:… GET /v2/dc23/blobs/sha256:… … Download config and

    layer blobs $ docker pull --platform linux/amd64 localhost:9001/dc23:latest latest: Pulling from dc23 2651927a96a6: Download complete 83df69d10500: Download complete 10e1614aca69: Download complete 725b720f91d7: Download complete Digest: sha256:5d0cbb3… Status: Image is up to date for localhost:9001/dc23:latest localhost:9001/dc23:latest
  27. Image Index (application/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … a9esta<on-manifest Config Blob Layer Layer attestation-manifest Config Blob Layer Layer 04 Pull
  28. 04 Pull HEAD /v2/dc23/manifests/<tag name> GET /v2/dc23/manifests/<image index digest> GET

    /v2/dc23/manifests/<image manifest digest> GET /v2/dc23/manifests/blobs/<config digest> GET /v2/dc23/manifests/blobs/<layer digest> GET /v2/dc23/manifests/blobs/<layer digest> … Requests → current digest of tag → image index json file → image manifest json file for the platform → config blob by its digest → layer blob by its digest → layer blob by its digest → …
  29. Pull linux/amd64 version of latest 1 04 Pull 1. Convert

    tag to digest 2. Select the image for the right platform 3. Download config and layer blobs
  30. Pull linux/amd64 version of latest 1 04 Pull 1. Convert

    tag to digest 2. Select the image for the right platform 3. Download config and layer blobs 1. Same Digest! 2. Manifests already downloaded 3. Blobs already downloaded
  31. 04 Pull HEAD /v2/dc23/manifests/<tag name> GET /v2/dc23/manifests/<image index digest> GET

    /v2/dc23/manifests/<image manifest digest> GET /v2/dc23/manifests/blobs/<config digest> GET /v2/dc23/manifests/blobs/<layer digest> GET /v2/dc23/manifests/blobs/<layer digest> … Requests → current digest of tag → image index json file → image manifest json file for the platform → config blob by its digest → layer blob by its digest → layer blob by its digest → …
  32. 04 Pull Different tags, same digest v2 repositories <repo name>

    _manifests tags latest current link index sha256 <digest> link 1 current link index sha256 <digest> link 1.0 ... blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  33. 05 New Version Let’s build a new image • Edit

    one single layer • Build on same and different tags
  34. Image Index (application/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … attestation-manifest Config Blob Layer Layer attestation-manifest Config Blob Layer Layer 05 New Version
  35. 05 New Version Registry View v2 repositories <repo name> _manifests

    tags latest current link index sha256 <digest> link <digest> link 1 … 1.0.0 current link index sha256 <digest> link 1.0.1 current link index sha256 <digest> link blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  36. 05 New Version Registry View v2 repositories <repo name> _manifests

    tags latest current link index sha256 <digest> link <digest> link 1 … 1.0.0 current link index sha256 <digest> link 1.0.1 current link index sha256 <digest> link blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  37. 05 New Version Registry View v2 repositories <repo name> _manifests

    tags latest current link index sha256 <digest> link <digest> link 1 … 1.0.0 current link index sha256 <digest> link 1.0.1 current link index sha256 <digest> link blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  38. 05 New Version Registry View v2 repositories <repo name> _manifests

    tags latest current link index sha256 <digest> link <digest> link 1 … 1.0.0 current link index sha256 <digest> link 1.0.1 current link index sha256 <digest> link blobs sha256 fe fe498ff… a8 a85ae31... fd fd03efd... ...
  39. Extend container images with related, non runnable, data OCI Artifacts

    Everywhere! Store other things than container image
  40. 06 Beyond Images Docker Compose as OCI Image Manifest {

    "schemaVersion": 2, "mediaType": "application/vnd.oci.image.manifest.v1+json", "artifactType": "application/vnd.docker.compose.project", "config": { "mediaType": "application/vnd.docker.compose.project", "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a", "size": 2, "annotations": { "com.docker.compose.version": "2.22.0" } }, "layers": [{ "mediaType": "application/vnd.docker.compose.file+yaml", "digest": "sha256:839ee3e27293c4f021ad49d8e71ec85bfc69706d1f06037b848a4f13564eeba8", "size": 343, "annotations": { "com.docker.compose": "2.22.0"
  41. 06 Beyond Images Homebrew as OCI Image Manifest { "mediaType":

    "application/vnd.oci.image.manifest.v1+json", "digest": "sha256:205f7a66495737af32db3125a63fc229622d8917b65eaf2436e4093f18948dc7", "size": 1911, "platform": { "architecture": "amd64", "os": "darwin", "os.version": "macOS 12" }, "annotations": { "org.opencontainers.image.ref.name": "2.12.1.monterey", "sh.brew.bottle.digest": "62534bceb8f7074827fa2146dd13603018aaf07c82e22cfef96571c8133ce8a1", "sh.brew.tab": "{\"homebrew_version\":\"3.4.11-152- ga3fab02\",\"changed_files\":[],\"source_modified_time\":1653865426,\"compiler\":\"clang\",\"runtime_ dependencies\":[],\"arch\":\"x86_64\",\"built_on\":{\"os\":\"Macintosh\",\"os_version\":\"macOS 12\",\"cpu_family\":\"penryn\",\"xcode\":\"13.4\",\"clt\":\"13.4.0.0.1.1651278267\",\"preferred_perl\ ":\"5.30\"}}" } }
  42. 06 Beyond Images CNAB Bundle { "schemaVersion": 2, "manifests": [

    { "mediaType": "application/vnd.oci.image.manifest.v1+json", "digest": "sha256:464e2efbee1cfa84d29b3305f0901c75dc70f2fa554cbcb7a342e21cf7d7f5e1", "size": 188, "annotations": { "io.cnab.manifest.type": "config" } }, { "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json", "digest": "sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913", "size": 2364, "annotations": { "io.cnab.manifest.type": "invocation" } } ], "annotations": { "io.cnab.runtime_version": "v1.0.0-WD", "io.docker.app.format": "cnab", "io.docker.type": "app", "org.opencontainers.artifactType": "application/vnd.cnab.manifest.v1"
  43. Image Index (application/vnd.oci.image.index.v1+json) linux/amd64 Config Blob Layer Layer … linux/arm64

    Config Blob Layer Layer … a9esta<on-manifest Config Blob Layer Layer attestation-manifest Config Blob Layer Layer 06 Beyond Images ?
  44. 06 Beyond Images OCI Image and Distribution Specs v1.1 •

    How to create and store alternative (even non container) artifacts • Manifest field for establishing relationships • Query relationships https://opencontainers.org/posts/blog/2023-07-07-summary-of- upcoming-changes-in-oci-image-and-distribution-specs-v-1-1/