DOCIR - Dynamic OCI Registry

OCI registries, like Distribution, Harbor and several other cloud-based registries, serve “static”, or pre-defined OCI (container) images. But what if we wanted to generate images dynamically?

A dynamic image is one that doesn’t need to be pre-defined, and can be composed on-the-fly based on a user’s preference expressed at image pull time. Not at the packager’s design time.

Think of what dynamic languages did for the web, but for container images. Enter a-la-carte container images.

Open a new world where users may have the exact container image that they want: with the base “OS” that they want, any number of components, minimal images or debugging images, with initial data or not, etc.

A Dynamic OCI Registry is not one that calls “docker build” behind the scenes; but rather one that has a “dynamic business logic” that is capable of composing layers from existing images and generating the appropriate manifests in real time.

Talk will include a live demo!

Original event URL: https://www.containerdays.io/containerdays-conference-2025/agenda/#sz-session-892858

Transcript

1. @ahachete Dynamic OCI Registry Container Days 2025 - Hamburg Alvaro Hernandez @ahachete
2. @ahachete ` whoami ` Alvaro Hernandez <aht@ongres.com> aht.es • Founder & CEO, OnGres • 20+ years Postgres user and DBA • Mostly doing R&D to create new, innovative software on Postgres • More than 140 tech talks, most about Postgres • Founder and President of the NPO Fundación PostgreSQL • AWS Data Hero
3. @ahachete Container image customization, variants… you name it
4. @ahachete Container (images) are immutable Torsten Bätge, Hamburg, CC BY-SA 3.0 via Wikimedia Commons
5. @ahachete Customization (image augmentation)
6. @ahachete Combinatorial explosion of image variants https://forums.rockylinux.org/t/on-the-container-images-what-do-the-different-variants-mean/16458/2
7. @ahachete Combinatorial explosion of image variants https://hub.docker.com/_/postgres
8. @ahachete Combinatorial explosion of image variants https://hub.docker.com/_/wordpress
9. @ahachete Combinatorial explosion of image variants https://learn.microsoft.com/en-us/dotnet/core/docker/container-images
10. @ahachete A little story about Postgres container images
11. @ahachete Postgres extensions pigsty.io/about/ext/
12. @ahachete Postgres extensions • Postgres extensions come from arbitrary, third-party repositories. • Users typically use several per instance. • It’s rare that no extensions are used at all. • There’s a long tail of what extensions do users use. • Some repos host 200-400 extensions. Total est: 1,000.
13. @ahachete How to pack Postgres extensions in container images?
14. @ahachete Option 1: the “fatty” container • Size • Security • Restarts (downtime) commons.wikimedia.org/wiki/File:Overloaded_Truck_%28105 22133955%29.jpg
15. @ahachete Option 2: dynamically inject into container (runtime) • Security • Startup time • Approach followed in StackGres See https://aht.es/#talks-postgres_extensions_in_kubernetes
16. @ahachete Formula to compute all possible number of images, considering n extensions to choose from Option 3: generate all possible container images
17. @ahachete Formula to compute all possible number of images, considering n extensions to choose from is bigger than Option 3: generate all possible container images
18. @ahachete n=10 → 252 n=200 → 9.05485146561032811654E+58 Option 3: generate all possible container images
19. @ahachete Option 4: dynamically generate container images
20. @ahachete D ynamic OCI R egistry
21. @ahachete First step: pack extensions as OCI images Why Postgres Extensions should be packaged and distributed as OCI images
22. @ahachete DOCIR: Dynamic OCI Registry • A new, from scratch, OCI Registry following the official specs. • Composes dynamic images on the fly: ◦ Adding (pre-existing) layers on-demand ◦ Generating dynamic manifests (index, image manifest, runtime config) • Dynamic business logic is custom: you decide how to compose layers • Similar to nixery.dev, but more general purpose
23. @ahachete How does it work? https://github.com /sudo-bmitch/pres entations/blob/mai n/oci-refers/img/o ci-image.png
24. @ahachete How does it work? https://github.com /sudo-bmitch/pres entations/blob/mai n/oci-refers/img/o ci-image.png
25. @ahachete How does it work? Dynamic runtime config
26. @ahachete How does it work? Dynamic manifest generation
27. @ahachete How to encode components in the image name? <name>:<reference> name [a-z0-9]+((\.|_|__|-+)[a-z0-9]+)*(\/[a-z0-9]+((\.|_|__|-+)[a-z0-9]+)*)* reference: [a-zA-Z0-9_][a-zA-Z0-9._-]{0,127} https://github.com/opencontainers/distribution-spec/blob/main/spec.md
28. @ahachete • It’s part of the “business logic” (i.e. custom) • Example: use an URL-like name ◦ Name: postgres/e/ext1/ext2/ext3 ◦ Reference: ignored ◦ With explicit versioning: postgres--16.3/e/ext1--v1/ext2--v2 • Potential limits to “URL” length! (e.g. Docker) The image “URL”
29. @ahachete DEMO !!
30. @ahachete Demo: pick some extension(s) randomly!
31. @ahachete DOCIR Today & Next steps
32. @ahachete DOCIR today • Written in modern Java (21) + Quarkus. • Dynamic generation is “custom business logic”, to be implemented. • Map from name:reference to array of layers and a runtime config. • Supports multi-arch dynamic container images, authentication token, S3 backend storage.
33. @ahachete DOCIR today: automatic metadata collection • Mapping name:reference to a list of layers is trivial for simple cases; but may otherwise require layer metadata. • DOCIR provides an interface for OCI metadata extraction. • And an implementation for storing metadata on a Postgres database. • Mapping business logic can talk to the database to determine the layers to add to the image.
34. @ahachete DOCIR: metadata name | pga/extension/cube-16.4 reference | 10-linux-amd64 id | 6179 digest | sha256:fe19652abbbe1fb4a1d87d668657ef3758a96441403dad43fd002c8bd3267d48 arch | amd64 arch_variant | os | linux title | cube 16.4 description | cube v16.4 for PostgreSQL 16.4 ... custom_labels | {"sh.pga.containers.type": "extension", "sh.pga.containers.homeURL": "https://postgresql.org", "sh.pga.containers.component.name": "cube", "sh.pga.containers.repository.tag": "REL_16_4", "sh.pga.containers.repository.url": "https://git.postgresql.org/git/postgresql.git", "sh.pga.containers.metadataVersion": "v1beta1", "sh.pga.containers.description.long": "Extension cube v16.4 for PostgreSQL 16.4", "sh.pga.containers.extension.flavor": "postgres", "sh.pga.containers.extensions.cube.tags": "contrib", "sh.pga.containers.extension.flavorVersion": "16.4", "sh.pga.containers.extensions.cube.install": "true", "sh.pga.containers.extensions.cube.trusted": "true", "sh.pga.containers.repository.commitDigest": "REL_16_4", "sh.pga.containers.extension.flavor.version": "16.4", "sh.pga.containers.extensions.cube.categories": "extension"}
35. @ahachete DOCIR tomorrow (next steps) • Upcoming v1.0 (Q3-Q4 2025). • Will be published as fully OSS under AGPLv3. • TODO: OCI distribution full compliance. • Open discussion to propose more flexibility for name:reference. • Bindings for custom logic in non Java languages?
36. @ahachete Dynamic OCI Registry Alvaro Hernandez @ahachete