Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build a container on Gitlab CI quest — Game Wal...

Build a container on Gitlab CI quest — Game Walkthrough

Examples available here: https://github.com/lig/gitlab-ci-docker

Serge Matveenko

November 22, 2019
Tweet

More Decks by Serge Matveenko

Other Decks in Technology

Transcript

  1. Quest A quest is a journey toward a specific mission

    or a goal. From Wikipedia, the free encyclopedia
  2. Why containers at all? Previously on Building a container image

    • Reproducible builds • Predicted deployments • OS (distro) independent • Unified packaging • Unified infrastructure • Scalability, orchestration, etc
  3. Building a container image Goals • Do not break the

    existing workflow • Container image is the primary build artifact • Predicted build behavior • Effective resources utilization • Time!
  4. Workflow • Know your workflow: git{,hub,lab} flow, release policy, etc

    • Use registry path for branches • Use image tags for tags • Use suffixes for build variants Naming container images
  5. Naming container images — git[lab] flow • branch:master — the

    main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags
  6. Naming container images — git[lab] flow • branch:master — the

    main integration branch registry.gitlab.com/<group>/<project>:latest before_script: - export CONTAINER_IMAGE="$CI_REGISTRY_IMAGE:latest"
  7. Naming container images — git[lab] flow • branch:master — the

    main integration branch • branch:<num-title> — feature/issue branches registry.gitlab.com/<group>/<project>/<branch>:latest before_script: - export CONTAINER_IMAGE="$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:latest"
  8. Naming container images — git[lab] flow • branch:master — the

    main integration branch • branch:<num-title> — feature/issue branches registry.gitlab.com/<group>/<project>[/<branch>]:latest
  9. Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:latest variables: GIT_REF_DEFAULT: master

    before_script: - export GIT_REF_SLUG - >- [[ $CI_COMMIT_REF_NAME = $GIT_REF_DEFAULT ]] && CONTAINER_REPO="$CI_REGISTRY_IMAGE" || CONTAINER_REPO="$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG" - export CONTAINER_REPO - export CONTAINER_IMAGE="$CONTAINER_REPO:latest"
  10. Naming container images — git[lab] flow • branch:master — the

    main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches registry.gitlab.com/<group>/<project>[/<branch>]:latest
  11. Naming container images — git[lab] flow • branch:master — the

    main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>]
  12. Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] variables: GIT_REF_DEFAULT: master

    before_script: - >- [[ -n "$CI_COMMIT_TAG" ]] && GIT_REF_SLUG="$(git branch -a --contains ${CI_COMMIT_TAG} | grep '^ remotes/origin/release-' | tail -c+18)" || GIT_REF_SLUG="$CI_COMMIT_REF_SLUG" - export GIT_REF_SLUG - >- [[ $CI_COMMIT_REF_NAME = $GIT_REF_DEFAULT ]] && CONTAINER_REPO="$CI_REGISTRY_IMAGE" || CONTAINER_REPO="$CI_REGISTRY_IMAGE/$GIT_REF_SLUG" - export CONTAINER_REPO - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" - export CONTAINER_RELEASE_IMAGE="$CONTAINER_REPO:$CI_COMMIT_TAG"
  13. Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] build images: script:

    - docker build -t ${CONTAINER_IMAGE} . push images: script: - docker push ${CONTAINER_IMAGE} only: - branches tag release images: script: - docker tag ${CONTAINER_IMAGE} ${CONTAINER_RELEASE_IMAGE} - docker push ${CONTAINER_RELEASE_IMAGE} only: - tags
  14. Naming container images — git[lab] flow • branch:master — the

    main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] done.
  15. Naming container images — build variants Dockerfile FROM python as

    base WORKDIR /usr/src/app RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . .
  16. Naming container images — build variants Dockerfile FROM python as

    base WORKDIR /usr/src/app RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . .
  17. registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>][-variant] before_script: - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" - export CONTAINER_RELEASE_IMAGE="$CONTAINER_REPO:$CI_COMMIT_TAG" build images:

    script: - docker build --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} . push images: script: - docker push ${CONTAINER_IMAGE} - docker push ${CONTAINER_IMAGE}-dev only: - branches Naming container images — build variants
  18. Resources • Optimize Dockerfiles • Local builder cache • Prepopulate

    cache • Use shared builder Utilizing build cache
  19. Utilizing build cache — Dockerfile FROM python as base WORKDIR

    /usr/src/app COPY ./Pipfile ./Pipfile.lock ./ RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . .
  20. Utilizing build cache — Local builder cache FROM python as

    base WORKDIR /usr/src/app COPY ./Pipfile ./Pipfile.lock ./ RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . .
  21. Utilizing build cache — Local builder cache Use `buildkit`. Build

    Enhancements for Docker — https://docs.docker.com/develop/develop-images/build_enhancements/
  22. Utilizing build cache — Prepopulate cache build images: stage: build

    script: - docker pull ${CONTAINER_IMAGE}-dev || true - docker pull ${CONTAINER_IMAGE} || true - docker build --cache-from ${CONTAINER_IMAGE}-dev --cache-from ${CONTAINER_IMAGE} --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} .
  23. Utilizing build cache — Use shared builder variables: DOCKER_HOST: tcp://shared-docker-builder.example.net:2375

    services: - docker:dind build images: stage: build script: - docker pull ${CONTAINER_IMAGE}-dev || true - docker pull ${CONTAINER_IMAGE} || true - docker build --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} .
  24. Extra • Use a hash while building a container instead

    of `latest` • Use buildkit if it works for you • Use LFS for big data and mount it instead of caring it with an image • Use multi-target builds Some do’s don'ts