Postgres à la carte: dynamic container images with your choice of extensions

When generating container images for Postgres, you also want to pack the desired extensions with it. But due to security (and size) reasons, you should not pack the hundreds of available extensions in the Postgres ecosystem, but only those that you would be using.

Since you can have an arbitrary selection of an arbitrary number of extensions from all the available catalog, having Postgres containers with just the selected extensions would require generating more container images than the number of atoms in the Universe.

Enter the world of dynamic OCI (container) images, a breakthrough technology that enables generating in real-time, on-the-fly, container images. Ever wanted à la carte Postgres container images? Now you can have Postgres and any choice of extensions that you want!

These images can be used for running Postgres on containers, including Kubernetes; or also on any other environment capable of consuming OCI images.

This talk will explore the motivations and technology behind dynamic container images, and how they are applied to solve the problem of adding any subset of extensions to the Postgres container images.

Transcript

1. Postgres à la carte Postgres à la carte: dynamic container images with your choice of extensions Álvaro Hernández @ahachete
2. Postgres à la carte @ahachete ` whoami ` Alvaro Hernandez <aht@ongres.com> aht.es • Founder & CEO, OnGres • 20+ years Postgres user and DBA • Mostly doing R&D to create new, innovative software on Postgres • More than 140 tech talks, most about Postgres • Founder and President of the NPO Fundación PostgreSQL • AWS Data Hero
3. Postgres à la carte Running Postgres on containers
4. Postgres à la carte @ahachete It’s as simple as it gets… $ docker run -d -e POSTGRES_PASSWORD=42 -p5432:5432 \ postgres (Docker’s official image)
5. Postgres à la carte …until you want what everybody wants:
6. Postgres à la carte @ahachete Extensions, extensions, extensions https://www.youtube.com/watch?v=EMldOiiG1Ko
7. Postgres à la carte @ahachete No worries, I’ll derive my own container image!
8. Postgres à la carte @ahachete No worries, I’ll derive my own container image! • Dozens of extensions come bundled with Docker’s official base image, whether you want them or not. • Adding more extensions typically means: ◦ Compiling from source ◦ Adding additional packages/libraries ◦ Knowing how to compile, package and clean build code/artifacts • Now you have become a container image maintainer! ◦ Updates are on you ◦ Security is on you
9. Postgres à la carte Call for Postgres + Extensions Distributions (specially on/for containers)
10. Postgres à la carte @ahachete Let’s pack extensions as OCI images! Why Postgres Extensions should be packaged and distributed as OCI images
11. Postgres à la carte Different options for packaging extensions within containers
12. Postgres à la carte @ahachete Option 1: the fatty container • Size • Security • Restarts (downtime)
13. Postgres à la carte @ahachete Option 2: dynamically inject into container • Security • Startup time • Approach followed in StackGres See https://aht.es/#talks-postgres_extensions_in_kubernetes
14. Postgres à la carte Option 3: dynamically generate container images
15. Postgres à la carte @ahachete Option 3: dynamically generate container images Formula to compute all possible number of images, considering n extensions to choose from
16. Postgres à la carte @ahachete Option 3: dynamically generate container images Formula to compute all possible number of images, considering n extensions to choose from
17. Postgres à la carte @ahachete Option 3: dynamically generate container images Formula to compute all possible number of images, considering n extensions to choose from is bigger than
18. Postgres à la carte @ahachete Option 3: dynamically generate container images Option 3: dynamically generate container images n=10 → 252 n=200 → 9.05485146561032811654E+58
19. Postgres à la carte @ahachete Option 3: dynamically generate container images DEMO
20. Postgres à la carte @ahachete Option 3: dynamically generate container images
21. Postgres à la carte @ahachete With a Web UI to construct the “URLs”
22. Postgres à la carte How does it work?
23. Postgres à la carte @ahachete How does it work? Credit: https://github.com /sudo-bmitch/pres entations/blob/mai n/oci-refers/img/o ci-image.png
24. Postgres à la carte @ahachete DOCIR: Dynamic OCI Registry • A new, from scratch, OCI Registry following the official specs. • Instead of serving static images, composes dynamic images on the fly: ◦ Adding (pre-existing) layers on-demand ◦ Generating dynamic manifests (index, image manifest, config) • Dynamic business logic is custom: you decide how to compose layers • Postgres specific version leverages image metadata to determine how to compose layers
25. Postgres à la carte @ahachete DOCIR: metadata name | pga/extension/cube-16.4 reference | 10-linux-amd64 id | 6179 digest | sha256:fe19652abbbe1fb4a1d87d668657ef3758a96441403dad43fd002c8bd3267d48 arch | amd64 arch_variant | os | linux title | cube 16.4 description | cube v16.4 for PostgreSQL 16.4 ... custom_labels | {"sh.pga.containers.type": "extension", "sh.pga.containers.homeURL": "https://postgresql.org", "sh.pga.containers.component.name": "cube", "sh.pga.containers.repository.tag": "REL_16_4", "sh.pga.containers.repository.url": "https://git.postgresql.org/git/postgresql.git", "sh.pga.containers.metadataVersion": "v1beta1", "sh.pga.containers.description.long": "Extension cube v16.4 for PostgreSQL 16.4", "sh.pga.containers.extension.flavor": "postgres", "sh.pga.containers.extensions.cube.tags": "contrib", "sh.pga.containers.extension.flavorVersion": "16.4", "sh.pga.containers.extensions.cube.install": "true", "sh.pga.containers.extensions.cube.trusted": "true", "sh.pga.containers.repository.commitDigest": "REL_16_4", "sh.pga.containers.extension.flavor.version": "16.4", "sh.pga.containers.extensions.cube.categories": "extension"}
26. Postgres à la carte @ahachete DOCIR: metadata
27. Postgres à la carte @ahachete Next steps • New release of image “layers” coming soon. • (Re)define metadata format. Integrate with PGXN v2 ? • Publish the website for container selection and REST API • Publish DOCIR as open source • Call for extension authors to join!
28. Postgres à la carte @ahachete Don’t miss “Reproducible Postgres” on Wed
29. Postgres à la carte Q & A