Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reusing Apps between Teams and Environments through Containers

Reusing Apps between Teams and Environments through Containers

Presentation shown in the WeAreDevelopers Container Day, Wednesday, February 3rd, 2021. Video: https://youtu.be/dudny94KWpo?t=3629

Adrian Kosmaczewski

February 03, 2021
Tweet

More Decks by Adrian Kosmaczewski

Other Decks in Technology

Transcript

  1. VSHN – The DevOps Company
    Adrian Kosmaczewski
    Sharing CLI Tools with
    Containers
    1

    View full-size slide

  2. VSHN – The DevOps Company
    Pronounced ˈvɪʒn – like "vision"
    Founded 2014 in Zürich
    Switzerland’s leading DevOps, Docker & Kubernetes
    expert with 24/7 support
    ISO 27001 certi ed & ISAE 3402 Report Type 1 veri ed
    First Swiss Kubernetes Certi ed Service Provider
    / /
    vshn.ch @vshn_ch vshn.tv
    2

    View full-size slide

  3. VSHN – The DevOps Company 3

    View full-size slide

  4. VSHN – The DevOps Company 4

    View full-size slide

  5. VSHN – The DevOps Company
    1. "Lightweight virtual machines"
    2. Docker Compose
    3. Kubernetes
    Containers at Work
    5

    View full-size slide

  6. VSHN – The DevOps Company
    Containers as a tool distribution mechanism
    Think Di erent
    6

    View full-size slide

  7. VSHN – The DevOps Company
    1. The Challenge of Internal Tools
    2. Encapsulating and Sharing Tools
    3. Reusing Tools in CI/CD Pipelines
    4. More Container Tool Ideas
    5. "Gotchas"
    Agenda
    7

    View full-size slide

  8. VSHN – The DevOps Company

    1. The Challenge of Internal Tools
    8

    View full-size slide

  9. VSHN – The DevOps Company 9

    View full-size slide

  10. VSHN – The DevOps Company
    retool.com/blog/state-of-internal-tools-2020
    10

    View full-size slide

  11. VSHN – The DevOps Company
    retool.com/blog/state-of-internal-tools-2020
    11

    View full-size slide

  12. VSHN – The DevOps Company
    Lots of
    Each VSHNeer
    Linux, Mac, Windows
    Choose the "best" programming language for the job
    Go, Java, Python, JavaScript, shell script
    VSHN
    internal tools
    can use their preferred OS
    12

    View full-size slide

  13. VSHN – The DevOps Company
    Go, Rust, C, C++
     Great for x-platform
    CLI tools!
     Complex to build
    Ruby, Python, JavaScript
     Agile: fast prototyping,
    great libraries
     Big runtimes to install
    and libraries to download
    Programming Languages
    13

    View full-size slide

  14. VSHN – The DevOps Company
    2. Encapsulating and Sharing Tools
    14

    View full-size slide

  15. VSHN – The DevOps Company
    TypeScript application
    npm download internet
    Example: "Greeter"
    gitlab.com/akosma/greeter
    15

    View full-size slide

  16. VSHN – The DevOps Company
    Greeter in Action
    $ podman run --rm quay.io/akosma/greeter:1.0 "WeAreDevelopers"
    _______________________
    < Hello WeAreDevelopers >
    -----------------------
    \ ^__^
    \ (oo)\_______
    (__)\ )\/\
    ||----w |
    || ||
    16

    View full-size slide

  17. VSHN – The DevOps Company
    Options:
    -V, --version output the version number
    -c, --character greeter (choices: "C3PO", "R2-D2", "ackbar", "ar
    "bunny", "cat", "default", "doge", "mona-lisa",
    -a, --action thing to do (choices: "talk", "think", default:
    -h, --help display help for command
    asciinema $ node out/src/index.js WeAreDevelopers!
    ________________________
    < Hello WeAreDevelopers! >
    ------------------------
    \ ^__^
    \ (oo)\_______
    (__)\ )\/\
    ||----w |
    || ||
    asciinema $
    00:00
    gitlab.com/akosma/greeter
    asciinema.org/a/lTo0VDVmMJyD6eF9tu56XLbeQ
    17

    View full-size slide

  18. VSHN – The DevOps Company
    Multi-Step Builds
    # Step 1: Builder image
    FROM node:14.15.1-alpine3.12 AS builder
    COPY [".eslintrc.js", ".eslintignore", "tsconfig.json", "gulpfile.js", "package.json",
    COPY src /command/src
    COPY spec /command/spec
    WORKDIR /command
    RUN npm install
    RUN npm test
    RUN npm run release
    # ..
    # Step 2: Runtime image
    FROM alpine:3.12
    RUN apk add --no-cache libstdc++
    COPY --from=builder /command/greeter.bin /usr/local/bin/greeter
    ENTRYPOINT [ "greeter" ]
    gitlab.com/akosma/greeter/-/blob/master/Docker le
    18

    View full-size slide

  19. VSHN – The DevOps Company
    asciinema $ podman build -t quay.io/akosma/greeter:1.0 .
    STEP 1: FROM node:14.15.1-alpine3.12 AS builder
    Completed short name "node" with unqualified-search registries (origin: /etc/co
    s.conf)
    Getting image source signatures
    Copying blob e75f007ed29c skipped: already exists
    Copying blob 05e7bc50f07f skipped: already exists
    Copying blob 94058c629e48 skipped: already exists
    Copying blob 6bd5f70f12c5 [--------------------------------------] 0.0b / 0.0b
    Copying config bc9a7579ff done
    Writing manifest to image destination
    Storing signatures
    00:00
    gitlab.com/akosma/greeter
    asciinema.org/a/qS0X1vBrZtMjHZhI8q55p4JfD
    19

    View full-size slide

  20. VSHN – The DevOps Company
    1. podman build -t quay.io/akosma/greeter:1.0 .
    2. podman images
    3. podman login quay.io
    4. podman push quay.io/akosma/greeter:1.0
    5.
    Recap'
    quay.io/repository/akosma/greeter
    20

    View full-size slide

  21. VSHN – The DevOps Company 21

    View full-size slide

  22. VSHN – The DevOps Company
    0:00
    22

    View full-size slide

  23. VSHN – The DevOps Company 23

    View full-size slide

  24. VSHN – The DevOps Company
    3. Reusing Tools in CI/CD Pipelines
    24

    View full-size slide

  25. VSHN – The DevOps Company
    .gitlab-ci.yml
    build:
    image: docker:19.03.12
    stage: build
    services:
    - docker:19.03.12-dind
    script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $CI_REGISTRY/akosma/greeter:latest .
    - docker push $CI_REGISTRY/akosma/greeter:latest
    gitlab.com/akosma/greeter/-/blob/master/.gitlab-ci.yml
    25

    View full-size slide

  26. VSHN – The DevOps Company
    Pipeline Run
    26

    View full-size slide

  27. VSHN – The DevOps Company
    Embedded Container Registry
    27

    View full-size slide

  28. VSHN – The DevOps Company
    $ docker login registry.gitlab.com
    ...
    $ podman login registry.gitlab.com
    ...
    28

    View full-size slide

  29. VSHN – The DevOps Company
    Reuse
    gitlab.com/akosma/fortune
    29

    View full-size slide

  30. VSHN – The DevOps Company
    Docker le
    # Step 1: print a greeting
    FROM registry.gitlab.com/akosma/greeter:latest
    RUN /usr/local/bin/greeter Dockerfile --character snoopy --action think
    # Step 2: build runtime image
    FROM python:3.7-alpine
    RUN apk add fortune
    WORKDIR /usr/src/app
    COPY requirements.txt ./
    RUN pip install --no-cache-dir -r requirements.txt
    COPY app.py /usr/src/app
    COPY templates /usr/src/app/templates/
    USER 1001
    EXPOSE 9090
    CMD [ "python", "app.py" ]
    gitlab.com/akosma/fortune/-/blob/master/Docker le
    30

    View full-size slide

  31. VSHN – The DevOps Company
    .gitlab-ci.yml
    stages:
    - build
    .base:
    stage: build
    image: docker:stable
    services:
    - docker:dind
    before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
    docker-build:
    extends: .base
    script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" .
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG"
    only:
    - master
    gitlab.com/akosma/fortune/-/blob/master/.gitlab-ci.yml
    31

    View full-size slide

  32. VSHN – The DevOps Company 32

    View full-size slide

  33. VSHN – The DevOps Company

    4. More Container Tool Ideas
    33

    View full-size slide

  34. VSHN – The DevOps Company
    Linting
    Black-box testing
    API documentation generation
    Code Tasks
    34

    View full-size slide

  35. VSHN – The DevOps Company
    Find
    using vale
    Create or
    Generate for docs
    for documentation
    Documentation Tasks
    dead links in HTML
    Check writing style
    Spell checking
    PDF EPUB
    search engine index
    Live preview
    35

    View full-size slide

  36. VSHN – The DevOps Company

    5. "Gotchas"
    36

    View full-size slide

  37. VSHN – The DevOps Company
    1. Creating CLI tools
    2. Building containers
    3. Programming language-speci c tips
    Types of Gotchas
    37

    View full-size slide

  38. VSHN – The DevOps Company
    Follow the
    Create man page for your tools using
    Pass secrets to tools as environment variables
    Document, document, document
    Use Make les to standardize projects
    5.1 Creating CLI Tools Gotchas
    Command Line Interface Guidelines
    Asciidoctor
    38

    View full-size slide

  39. VSHN – The DevOps Company
    pages := $(shell find . -type f -name '*.adoc')
    out_dir := ./docs
    docker_cmd ?= podman
    docker_opts ?= --rm --tty # add --user "$$(id -u)" if using docker!
    antora_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}":/antora antora/anto
    vale_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}"/docsrc/modules/ROOT/pa
    hunspell_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}":/spell vshn/hunspe
    preview_cmd ?= $(docker_cmd) run --rm --publish 35729:35729 --publish 2020:2020 --volu
    htmltest_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}"/$(out_dir):/test w
    .PHONY: all
    all: html
    .PHONY: clean
    clean:
    rm -rf $(out_dir)
    .PHONY: vale
    vale:
    $(vale_cmd)
    .PHONY: spell
    spell: html
    $(hunspell cmd)
    39

    View full-size slide

  40. VSHN – The DevOps Company
    Beware of base images using musl instead of glibc
    …like Alpine!
    Free some disk space regularly with podman image prune
    ENTRYPOINT > CMD
    COPY > ADD
    Use other container registries
    Inspect your images using
    5.2 Gotchas when Building
    Containers
    dive
    40

    View full-size slide

  41. VSHN – The DevOps Company
    Self-hosted: , , , and
    SaaS: , , ,
    ( ), and ( )
    Embedded: and
    Container Registries
    kraken Harbor Docker distribution
    Sonatype Nexus
    Docker Pro or Team plans Quay AWS ECR GitHub
    Packages ghcr.io Google Container Registry gcr.io
    OpenShift GitLab
    41

    View full-size slide

  42. VSHN – The DevOps Company
    1. Python
    2. Go
    3. JavaScript
    4. Shell scripts
    5.3 Language-speci c tips
    42

    View full-size slide

  43. VSHN – The DevOps Company
     Use virtual environments!
    5.3.1 Python Tips
    # During development...
    $ python3 -m venv .venv
    $ source .venv/bin/activate
    $ pip install PyYAML
    $ pip install PyGithub
    $ pip freeze > requirements.txt
    # In the Dockerfile...
    RUN pip install --no-cache-dir -r requirements.txt
    43

    View full-size slide

  44. VSHN – The DevOps Company
     Use the base image!
    5.3.2 Go Tips
    distroless
    44

    View full-size slide

  45. VSHN – The DevOps Company
    Use TypeScript!
    npm install --prod
    Use to create standalone binaries
    Use the base image for minimalistic, small
    nal container images ( ).
    5.3.3 JavaScript Tips
    pkg
    scratch-node
    GitHub
    45

    View full-size slide

  46. VSHN – The DevOps Company
    Use
    And if you use Visual Studio Code, install the
    Beware of base images that do not contain /bin/bash and
    only have /bin/sh (like with )
    Either make sure your shell scripts work with /bin/sh,
    or install bash in your target image
    5.3.4 Shell Scripting Tips
    ShellCheck
    ShellCheck VS extension
    Alpine Busybox
    46

    View full-size slide

  47. VSHN – The DevOps Company
    Wrap command line tools in containers
    Use a public or private registries to share your tools
    Document them properly: README + man page + Antora
    website + …
    Beware of common issues
    Summary
    47

    View full-size slide

  48. VSHN – The DevOps Company
    Christian Cremer
    Tobias Brunner
    Aarno Aukia
    Acknowledgements
    ccremer on GitHub
    @tobruzh on Twitter
    @aarnoaukia on Twitter
    48

    View full-size slide

  49. VSHN – The DevOps Company
    Adrian Kosmaczewski – –
    VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 –
    – –
    Thanks!
    @akosma [email protected]
    [email protected]
    vshn.ch @vshn_ch vshn.tv
    49

    View full-size slide