Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Docker Image Packaging Pipeline Usin...

Building a Docker Image Packaging Pipeline Using GitHub Actions

DockerCon 2020 talk on creating reusable Dockerfiles, and building a packaging pipeline using GitHub Actions.

Gareth Rushgrove

May 28, 2020
Tweet

More Decks by Gareth Rushgrove

Other Decks in Programming

Transcript

  1. What does Dockerfile do? Create a filesystem, add metadata #

    Use the official image as a parent image. FROM node:current-slim # Set the working directory. WORKDIR /usr/src/app # Copy the file from your host to your current location. COPY package.json . # Run the command inside your image filesystem. RUN npm install # Inform Docker that the container is listening on the specified port at runtime. EXPOSE 8080 # Run the specified command within the container. CMD [ "npm", "start" ] # Copy the rest of your app's source code from your host to your image filesystem. COPY . .
  2. Time passes... Something has gone terrible wrong Dockerfile Dockerfile Dockerfile

    Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile
  3. What can we do? Let’s start with a very simple

    Dockerfile FROM alpine COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest
  4. Docker build args Build-time variables with default values FROM alpine

    COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest
  5. Pass values at build time Can pass from local ENV

    as well FROM alpine COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest $ docker build --build-arg VERSION=v0.19.0
  6. Just keep adding ARGs! Possible to build very generic Dockerfiles

    FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest FROM alpine ARG LABEL=v0.18.2 ARG IMAGE=open-policy-agent/conftest ARG PATH=/conftest COPY --from=$IMAGE:$LABEL $PATH $PATH
  7. Define workflows in YAML Stored in your repository at .github/workflows

    name: .NET Core on: push jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Setup .NET Core uses: actions/setup-dotnet@v1 with: dotnet-version: 3.1.101 - name: Install dependencies run: dotnet restore - name: Build run: dotnet build --configuration Release --no-restore - name: Test run: dotnet test --no-restore --verbosity normal
  8. GitHub Actions uses Docker Standard docker commands just work name:

    Docker on: push: branches: [ master ] pull_request: branches: [ master ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Build the Docker image run: docker build . --file Dockerfile --tag my-image-name:$(date +%s)
  9. build-push-action basics Build and push an image to Docker Hub

    uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tags: latest
  10. GitHub Actions supports logical operations Push only on tags, rather

    than any commit uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tag_with_ref: true push: ${{ startsWith(github.ref, 'refs/tags/') }}
  11. Lots of built-in goodies Lower the barrier to entry for

    good practice uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tag_with_ref: true tag_with_sha: true add_git_labels: true always_pull: true labels: label_name_1=label_value_1 target: mytarget
  12. Find security vulnerabilities in your applications Note the :golang tag

    on snyk/snyk $ git clone [email protected]:puppetlabs/wash.git $ docker run --rm -it --env SNYK_TOKEN -v $(PWD):/app snyk/snyk:golang Testing /app... Organization: garethr Package manager: gomodules Target file: go.mod Open source: no Project path: /app Licenses: enabled ✓ Tested 426 dependencies for known issues, no vulnerable paths found. Next steps: - Run `snyk monitor` to be notified about new related vulnerabilities. - Run `snyk test` as part of your CI/test.
  13. Runs on push and on a regular schedule Nice way

    of making sure we rebuild regularly name: Build and push images on: push: branches: - master paths: - "*" - "!README.md" - "!build.rb" schedule: # As well as running when we make changes we should run at least # every week in order to pick up new parent images and new versions of Snyk - cron: "0 0 * * 0"
  14. Runs on push and on a regular schedule Nice way

    of making sure we rebuild regularly name: Build and push images on: push: branches: - master paths: - "*" - "!README.md" - "!build.rb" schedule: # As well as running when we make changes we should run at least # every week in order to pick up new parent images and new versions of Snyk - cron: "0 0 * * 0" Don’t rebuild on documentation changes
  15. 1 Dockerfile Take an image as an argument and set

    some defaults ARG IMAGE FROM ${IMAGE} as parent WORKDIR /app COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] CMD ["snyk", "test"] FROM ubuntu as snyk RUN apt-get update && apt-get install -y curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep linux | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-linux.sha256 && \ mv snyk-linux /usr/local/bin/snyk && \ chmod +x /usr/local/bin/snyk
  16. 1 Dockerfile Download precompiled Snyk for each platform FROM ubuntu

    as snyk RUN apt-get update && apt-get install -y curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep linux | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-linux.sha256 && \ mv snyk-linux /usr/local/bin/snyk && \ chmod +x /usr/local/bin/snyk FROM alpine as snyk-alpine RUN apk add --no-cache curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep alpine | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-alpine.sha256 && \ mv snyk-alpine /usr/local/bin/snyk && \
  17. 1 Dockerfile Output targets for each platform FROM parent as

    alpine RUN apk add --no-cache libstdc++ COPY --from=snyk-alpine /usr/local/bin/snyk /usr/local/bin/snyk FROM parent as linux COPY --from=snyk /usr/local/bin/snyk /usr/local/bin/snyk
  18. Runs on push Run only when build files change name:

    "Generate Actions to build Snyk images" on: push: branches: - master paths: - build.rb - linux - alpine
  19. Enter the matrix Generated matrix for each combination matrix: include:

    - base: clojure:boot tag: clojure-boot target: linux - base: clojure:lein tag: clojure-lein target: linux - base: clojure:tools-deps tag: clojure-tools-deps target: linux - base: golang tag: golang target: linux - base: golang:1.12
  20. build-push-image With the inputs from the matrix steps: - uses:

    actions/checkout@v1 - uses: docker/build-push-action@v1 env: DOCKER_BUILDKIT: "1" with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} add_git_labels: true target: ${{ matrix.target }} repository: snyk/snyk tags: ${{ matrix.tag }} build_args: IMAGE=${{ matrix.base }}
  21. Example OCI labels Great for understanding provenance $ docker inspect

    snyk/snyk:python --format="{{json .Config.Labels}}" | jq { "org.opencontainers.image.created": "2020-05-17T00:14:15Z", "org.opencontainers.image.revision": "fa19b4d7fda74be0f342926c3c7feeb368f88b17", "org.opencontainers.image.source": "https://github.com/snyk/snyk-images" }
  22. Learn Dockerfile Dockerfile is incredible simple to get started with,

    but hides some powerful features that not enough folks learn to use
  23. Focus on maintenance The value of reducing the overhead of

    ongoing work needed to do the job is often underestimated