Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CyberAgent AI事業本部MLOps研修Container編 / Container ...

chck
June 22, 2023

CyberAgent AI事業本部MLOps研修Container編 / Container for MLOps

事業部研修のMLOps - Container編の資料です

chck

June 22, 2023
Tweet

More Decks by chck

Other Decks in Technology

Transcript

  1. ©2023 CyberAgent Inc. Distribution prohibited Today’s Schedule 10:00 - 12:30

    … Container編 13:45 - 14:45 … MLOps基礎編 15:00 - 18:30 … MLOps応用編
  2. ©2023 CyberAgent Inc. Distribution prohibited Today’s Schedule 10:00 - 12:30

    … Container編 13:45 - 14:45 … MLOps基礎編 15:00 - 18:30 … MLOps応用編
  3. ©2023 CyberAgent Inc. Distribution prohibited Prerequisites 6 1. Docker Desktop

    2. Docker Hub ユーザ登録 3. pyenv 4. poetry 5. awscli (認証含む) 6. hey 7. direnv 8. Terraform
  4. ©2023 CyberAgent Inc. Distribution prohibited Yuki Iwazaki 2014...Backend Engineer in

    DSP └2017...ML/DS in Inhouse └2018-...Research Engineer in AI Lab Multimedia (Vision & Language) 7 @chck
  5. ©2023 CyberAgent Inc. Distribution prohibited Docker Docker is an open

    platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production. https://docs.docker.com/get-started/overview/ 11
  6. ©2023 CyberAgent Inc. Distribution prohibited Why Docker? コードを動かすための依存をDockerさえ入っていれば動く状態に • PipやConda、Virtualenvではだめ?

    • 移行や共有時どうする? • OSへの直Installや別途Downloadが必要な依存がある場合は? 実験コードを動かすための長く複雑な手順をDockerで隠蔽できる • READMEにはDocker commandを書くだけ Kubernetesを始めとする強力なContainer ServiceにML AppをDeployできる • 2023年現在、一定規模以上のAppの運用を考えると業界標準に 12
  7. ©2023 CyberAgent Inc. Distribution prohibited Live Demo ➔ git clone

    https://github.com/chck/container4ml-aws.git ➔ cd container4ml-aws ➔ ls -a .dockerignore .envrc .github 2-jupyter infra Makefile .env.example .git 1-simple 3-fastapi LICENSE README.md ➔ cp .env.example .env 14
  8. ©2023 CyberAgent Inc. Distribution prohibited 初期設定 (.env) ➔ $EDITOR .env

    -------------------------- AWS_BUCKET= AWS_REGION=ap-northeast-1 AWS_ACCOUNT_ID= USER= ➔ direnv allow . 用意したAWSアカウントID USERは任意のIDを入れる .envが環境変数に反映される 用意したS3 BUCKET
  9. ©2023 CyberAgent Inc. Distribution prohibited 初期設定 (Terraform) ➔ make bucket

    ➔ cd infra ➔ terraform init ➔ terraform plan ➔ terraform apply ➔ cd .. 用意したAWSアカウントにTerraform用のbucketを作成 作られるResourceの確認 用意したAWSアカウントに対しTerraformを実行
  10. ©2023 CyberAgent Inc. Distribution prohibited Story 1 既にあるPython ScriptをDocker化したい Docker化のProcess

    • Python Scriptを動かすための依存を含んだDocker Imageを作成 • Docker Containerとして実行し、挙動を確認 • DockerfileのCommitやDocker RegistryにImageをUploadして完了 17
  11. ©2023 CyberAgent Inc. Distribution prohibited Docker Image An image is

    a read-only template with instructions for creating a Docker container. Often, an image is based on another image, with some additional customization. https://docs.docker.com/get-started/overview/ 18
  12. ©2023 CyberAgent Inc. Distribution prohibited Docker Container A container is

    a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container 19
  13. ©2023 CyberAgent Inc. Distribution prohibited Docker Container A container is

    a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container MacBook, ThinkPad, EC2, GCE macOS, Windows, Ubuntu JupyterLab, Flask, MySQL, Original App 20
  14. ©2023 CyberAgent Inc. Distribution prohibited Story 1: 再掲 既にあるPython ScriptをDocker化したい

    Docker化のProcess • Python Scriptを動かすための依存を含んだDocker Imageを作成 • Docker Containerとして実行し、挙動を確認 • DockerfileのCommitやDocker RegistryにImageをUploadして完了 21
  15. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ cd 1-simple

    ➔ ls .dockerignore Dockerfile Makefile pyproject.toml compose.yml main.py poetry.lock ➔ $EDITOR Dockerfile -------------------------- 23
  16. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile

    -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 24
  17. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile

    -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 25
  18. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile

    -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] Base Imageを指定 26
  19. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile

    -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 環境変数の定義 27
  20. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile

    -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 環境構築の実行コマンドを記述 28
  21. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile

    -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] Image内でのDirectory指定 29
  22. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile

    -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] local://${pwd}/main.py docker://app/main.py 30
  23. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile

    -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 起動時に実行されるコマンド 31
  24. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Imageの作成 ➔ docker

    build . -t container4ml-simple:1.0 コンテキストの指定. Dockerに見てほしいpath. COPY句でどこを起点にするか . 基本的にDockerfileのある場所でOK イメージ名. お作法的にはowner/image_name イメージタグ. イメージは更新されゆくのでいわゆる versioning 33
  25. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Imageの作成 ➔ docker

    build . -t container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE container4ml-simple 1.0 $(IMAGE_ID) 7 seconds ago 628MB Image sizeはなるべく小さくするべきだが、 慣れるまではゴリゴリに削らなくて OK 34
  26. ©2023 CyberAgent Inc. Distribution prohibited 4. Docker Containerの起動 ➔ watch

    docker ps -a ➔ docker run container4ml-simple:1.0 scikit-learn: 1.2.2 ➔ docker run --rm container4ml-simple:1.0 ➔ cat main.py import sklearn print(f"scikit-learn: {sklearn.__version__}") 35 ←watchとは別窓で実行
  27. ©2023 CyberAgent Inc. Distribution prohibited 4. Docker Containerの実行上書き ➔ tail

    -1 Dockerfile ➔ docker run --rm container4ml-simple:1.0 ls -lh total 4.0K -rw-r--r-- 1 root root 62 May 8 23:52 main.py 実行コマンドは上書きできる 同Imageで挙動だけ変えたい時等で活用 例えばTraining/ServingのImageを共通化するとか 36
  28. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ open

    https://hub.docker.com/ ➔ docker login ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/$(HUB_ID)/container4ml-simple ➔ docker rmi $(HUB_ID)/container4ml-simple:1.0 ➔ docker pull $(HUB_ID)/container4ml-simple:1.0 ➔ docker images 37
  29. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ open

    https://hub.docker.com/ ➔ docker login ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB 38
  30. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ open

    https://hub.docker.com/ ➔ docker login ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB 39 TagはAliasとして働くので同じImage IDを持ち, Diskも重複消費しない
  31. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ docker

    push $(HUB_ID)/container4ml-simple:1.0 ➔ open hub.docker.com/repository/docker/$(HUB_ID)/container4ml-simple ➔ docker rmi $(HUB_ID)/container4ml-simple:1.0 ➔ docker pull $(HUB_ID)/container4ml-simple:1.0 ➔ docker images 指定registryにimageをupload 40
  32. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ docker

    push $(HUB_ID)/container4ml-simple:1.0 ➔ open hub.docker.com/repository/docker/$(HUB_ID)/container4ml-simple ➔ docker rmi $(HUB_ID)/container4ml-simple:1.0 ➔ docker pull $(HUB_ID)/container4ml-simple:1.0 ➔ docker images 41 指定したDocker Imageを削除. Registryに上げていれば安心 指定RegistryからImageをDownload
  33. ©2023 CyberAgent Inc. Distribution prohibited Tips: Image Tagは複数付与できる ➔ docker

    tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:latest ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ docker push $(HUB_ID)/container4ml-simple:latest 42
  34. ©2023 CyberAgent Inc. Distribution prohibited Tips: slim? alpine? ➔ docker

    pull python:3.9 ➔ docker pull python:3.9-slim ➔ docker pull python:3.9-alpine ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE python 3.9 67ec76d9f73b 2 weeks ago 857MB python 3.9-slim 64458f531a7e 2 weeks ago 118MB python 3.9-alpine d314e28e240c 2 days ago 57.8MB 依存とサイズのトレードオフ 最初はslimがおすすめ 43 最軽量版 (alpine) 軽量版 (slim) Debian baseのfull image (無印)
  35. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 44 Docker Client:

    CUIとしてコマンドで操作 https://docs.docker.com/get-started/overview/#docker-architecture
  36. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 45 Docker Host:

    仮想環境の本体 DockerのContainerやImage, NetworkやVolumeを管理 Clientのコマンドを待受 https://docs.docker.com/get-started/overview/#docker-architecture
  37. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 46 Docker Registry:

    Docker Imageの共有先 PublicとPrivateを使い分け https://docs.docker.com/get-started/overview/#docker-architecture
  38. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 48 こっちは Remote

    Server Docker Hubや GCR, ECRが相当 https://docs.docker.com/get-started/overview/#docker-architecture
  39. ©2023 CyberAgent Inc. Distribution prohibited Story 2 学習コードの乗ったJupyterLabをDocker上で動かしたい Docker Compose化のProcess

    • Dockerfileを書くところまで同じ • compose.ymlを作成 • Docker Composeによる実行、挙動の確認 49
  40. ©2023 CyberAgent Inc. Distribution prohibited Docker Compose Compose is a

    tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. 50 https://docs.docker.com/compose/
  41. ©2023 CyberAgent Inc. Distribution prohibited Docker vs Docker Compose Docker:

    Docker Compose: 51 ➔ docker build . -t container4ml ➔ docker run --rm -p 8888:8888 -v ${PWD}:/app container4ml (compose.ymlがある状態で) ➔ docker compose up ↓概念の説明なので実行しなくて OK
  42. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く ➔ cd 2-jupyter

    ➔ ls .dockerignore compose.yml Makefile pyproject.toml .gitignore Dockerfile poetry.lock train.ipynb ➔ $EDITOR Dockerfile -------------------------- 52
  43. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く 53 ➔ $EDITOR

    Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]
  44. ©2023 CyberAgent Inc. Distribution prohibited 1-simpleのDockerfileと比較 ➔ $EDITOR Dockerfile --------------------------

    FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 54
  45. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く 55 ➔ $EDITOR

    Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]
  46. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml

    -------------------------- services: jupyter: image: …/container4ml-jupyter:${USER} build: . ports: - "8888:8888" volumes: - ${PWD}:/app 56
  47. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml

    -------------------------- services: jupyter: image: …/container4ml-jupyter:${USER} build: . ports: - "8888:8888" volumes: - ${PWD}:/app 57 任意のService名 Dockerfileを参照しながら 指定Image名でdocker build portやvolume optionを付与してdocker runされ jupyter containerが起動
  48. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker

    compose) ➔ docker compose up Jupyter Server 2.5.0 is running at:... ➔ docker images ➔ docker ps 58 ←docker compose upとは別窓で実行 ←docker buildとdocker runが両方行われる
  49. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker

    compose) ➔ open http://127.0.0.1:8888 ➔ train.ipynbを開いてRun -> Run All Cells └ ua_classifier.binが作られたことを確認 59
  50. ©2023 CyberAgent Inc. Distribution prohibited User Agent (UA) https://towardsdatascience.com/still-parsing-user-agent-strings-for-your-machine-learning-models-use-this-instead-8928c0e7e74f Client

    (e.g. ブラウザ) がServerアクセス時に付与するOS等の情報をまとめた文字列 Web広告の文脈では擬似的な個人情報として使えたりするため昨今規制が厳しい 60
  51. ©2023 CyberAgent Inc. Distribution prohibited Tips: ModelをCloud Storageに保存しておく例 ➔ ts

    = !TZ=Asia/Tokyo date +"%Y%m%d%H%M" ➔ aws s3 cp ua_classifier.bin s3://${AWS_BUCKET}/models/ua_classifier/{ts[0]}.bin ➔ aws s3 cp ua_classifier.bin s3://${AWS_BUCKET}/models/ua_classifier/latest.bin ➔ aws s3 ls --human-readable s3://${AWS_BUCKET}/models/ua_classifier/ 2023-05-02 08:19:44 281.9 KiB 202305020819.bin 2023-05-02 08:19:50 281.9 KiB latest.bin ↓Tipsなので実行しなくてOK
  52. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker

    compose) ➔ Ctrl-C └ Gracefully stopping... ➔ docker compose build ➔ docker run …/container4ml-jupyter:${USER} ls └ ua_classifier.bin ➔ aws ecr get-login-password --region ${AWS_REGION} | docker login -u AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker push …/container4ml-jupyter:${USER} └ Amazon ECRのUIを確認 63
  53. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く ➔ cd 3-fastapi

    ➔ ls bin Dockerfile main.py pyproject.toml compose.yml k8s poetry.lock setup.cfg ➔ $EDITOR Dockerfile -------------------------- 65
  54. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く 66 ➔ $EDITOR

    Dockerfile -------------------------- ARG AWS_ACCOUNT_ID ARG AWS_REGION=${AWS_REGION:-"ap-northeast-1"} ARG TRAINER_VERSION=${TRAINER_VERSION:-"latest"} FROM ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-jupyter:${TRAINER_VERSION} AS trainer FROM python:3.9-slim ...(中略)... WORKDIR /models COPY --from=trainer /app/ua_classifier.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]
  55. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く 67 ➔ $EDITOR

    Dockerfile -------------------------- ARG AWS_ACCOUNT_ID ARG AWS_REGION=${AWS_REGION:-"ap-northeast-1"} ARG TRAINER_VERSION=${TRAINER_VERSION:-"latest"} FROM ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-jupyter:${TRAINER_VERSION} AS trainer FROM python:3.9-slim ...(中略)... WORKDIR /models COPY --from=trainer /app/ua_classifier.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]
  56. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml

    -------------------------- services: myapp: image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 68
  57. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml

    -------------------------- services: myapp: image: …/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 69 任意のService名 Dockerfileを参照しながら 指定Image名でdocker build 指定Imageをdocker pullで用意 docker runのoption相当 myredis containerの起動を待ってから myappを起動
  58. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml

    -------------------------- services: myapp: image: …/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 70 w/o docker compose ➔ docker network create mynwk ➔ docker run -p 3333:80 -e PYTHONUNBUFFERED=1 -e DEBUG=true -e REDISHOST=myredis -e MODEL_NAME=A -v ${PWD}:/app --net mynwk …/container4ml-fastapi:latest ➔ docker run --net mynwk redis:alpine w/ docker compose ➔ docker compose up Container間通信(mynwk) myapp myredis
  59. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker

    compose) ➔ aws ecr get-login-password --region ${AWS_REGION} | docker login -u AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker compose up -d ➔ docker ps CONTAINER ID IMAGE ... PORTS ... NAMES ... .../container4ml-fastapi:1.0 ... 0.0.0.0:3333->80/tcp 3-fastapi-myapp-1 ... redis:alpine ... 6379/tcp 3-fastapi-myredis-1 ➔ open http://localhost:3333 ➔ open http://localhost:3333/stats ➔ docker compose down ➔ compose.ymlのMODEL_NAMEをAからBに変更後、 docker compose up --build -d 72
  60. ©2023 CyberAgent Inc. Distribution prohibited 時間が余った時用 - docker system prune

    -a - M1/M2 MacのPython開発事情 - 1 Image : 1 Appの理由 - Docker化が嬉しい場面 - Dataなどの重たいファイルはvolumeでsyncしておく話 - ModelをStorageに持つかImageに持つか - Docker container のDebug方法 - 変数によってbuildの振る舞いを変えたい時 - Training/Servingは同じImageか分けるか - Securityの話 73
  61. ©2023 CyberAgent Inc. Distribution prohibited Container Orchestration Containerの展開や状態の 管理を担うService Docker

    ImageをどうServing するかの部分 Kubernetesを基とする Managed Serviceが複数展開 https://www.datadoghq.com/container-report-2020/ 80
  62. ©2023 CyberAgent Inc. Distribution prohibited Docker Compose vs Kubernetes ➔

    docker compose up -d ➔ docker compose ps ➔ docker compose down ➔ cd 3-fastapi/k8s ➔ kind create cluster --config=kind-config.yml ➔ kubectl cluster-info --context kind-kind ➔ kubectl apply -f deployment.yml ➔ kubectl apply -f service.yml ➔ kubectl get po,svc,deploy ➔ kubectl delete -f service.yml ➔ kubectl delete -f deployment.yml ➔ kind delete cluster
  63. ©2023 CyberAgent Inc. Distribution prohibited AWS App Runner AWSが提供する Managed

    Container Service k8sの細かい所をwrapして AppのCodingに集中できる ようにしたもの 86
  64. ©2023 CyberAgent Inc. Distribution prohibited Story 3-2 LocalのDocker Composeで動作確認のできたUA Classifierを

    App RunnerにDeployしたい DeployのProcess 1. Docker ImageをRegistryにUpload 2. UploadしたImageを指定してApp Runnerを構築・起動 87
  65. ©2023 CyberAgent Inc. Distribution prohibited 1. RegistryにImageをUpload ➔ docker tag

    ${HUB_ID}/container4ml-fastapi:1.0 ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:1.0 ➔ docker tag ${HUB_ID}/container4ml-fastapi:latest ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest ➔ aws ecr get-login-password --region ap-northeast-1 | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:1.0 ➔ docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest ➔ open https://ap-northeast-1.console.aws.amazon.com/ecr/repositories 89
  66. ©2023 CyberAgent Inc. Distribution prohibited Autoscaling https://github.com/rakyll/hey WebAppの負荷試験ができるheyでbenchmarkを取ってみる ➔ sh

    ./test-scale.sh ➔ sh ./test-scale.sh https://${APPRUNNER_DOMAIN} App Runnerのmetricsやheyのbenchmarkを観察 96
  67. ©2023 CyberAgent Inc. Distribution prohibited GitHub ActionsによるDeployの自動化 ➔ cat .github/workflows/deploy-container4ml.yml

    -------------------------- ... on: push: branches: [main] ... jobs: infra: runs-on: ubuntu-latest steps: - run: echo "container4ml deploy job triggered." changes: runs-on: ubuntu-latest needs: infra ... トリガ条件 jobs ∋ steps needsで実行順を制御できる 実際に実行されるのは stepsのコマンド
  68. ©2023 CyberAgent Inc. Distribution prohibited ➔ cat .github/workflows/deploy-container4ml.yml -------------------------- ...

    on: push: branches: [main] ... jobs: infra: runs-on: ubuntu-latest steps: - run: echo "container4ml deploy job triggered." changes: runs-on: ubuntu-latest needs: infra ... GitHub ActionsによるDeployの自動化 GitHub repo -> Actionsから実行履歴が見れる
  69. ©2023 CyberAgent Inc. Distribution prohibited GitHub ActionsによるDeployの自動化 トリガ条件: main branchにpushされたら

    実行内容: paths-filterで更に分岐 └ trainerルート... '2-jupyter/**' の差分 └ S3からua_classifier.binをdownloadし、それを含めて       container4ml-jupyterをdocker build、ECRにpush └ predictorルート... '3-fastapi/**' の差分 └ container4ml-fastapiをdocker build、ECRにpush
  70. ©2023 CyberAgent Inc. Distribution prohibited Blue-green deployment 既存のApp (Blue) 稼働中の裏に

    新版のApp (Green) をTrafficが来ない 状態でDeployし、Standbyになった タイミングでRouter内部の向き先を Greenに変えることで無停止Deployを 行う仕組み App Runnerもこれに対応 ➔ sh ./test-switch.sh https://${APPRUNNER_DOMAIN} ➔ # Deploy New Version in App Runner 101 https://candost.blog/the-blue-green-deployment-strategy/
  71. ©2023 CyberAgent Inc. Distribution prohibited Traffic Splitting BlueとGreenで m%(A):n%(B)にRequestを分配 この仕組みで

    新機能のCanary Releaseや A/B Testも可能 102 https://dev.classmethod.jp/articles/alb-blue-green-deployment/
  72. ©2023 CyberAgent Inc. Distribution prohibited Simple A/B Testing Version A

    と BでTrafficを分け、Metricsを計測 有意差を確認 104 Version:1.0 Version:1.1 Routing 50% 50%
  73. ©2023 CyberAgent Inc. Distribution prohibited A/B Testingの実装方法 Load balancerでSplitting Container

    A, Bの前段にLoad balancerを立ててm:nに分散 - Pros 👍 ??? - Cons 👎 ??? 105 Application内でSplitting Python Code内でrandomやuuidのhash値、 DB参照などのルールに基づいてif else等で分割 - Pros 👍 ??? - Cons 👎 ??? ざっくり2種類の方法がある
  74. ©2023 CyberAgent Inc. Distribution prohibited A/B Testingの実装方法 Load balancerでSplitting Container

    A, Bの前段にLoad balancerを立ててm:nに分散 - Pros 👍 ModelとContainerが1:1で紐づくので Appの実装がシンプル - Cons 👎 Cache戦略によっては同じuuidでも Requestの度にA/Bを横断してしまう 106 Application内でSplitting Python Code内でrandomやuuidのhash値、 DB参照などのルールに基づいてif else等で分割 - Pros 👍 QueryのUUIDに応じて分割先のルール設計が簡単 - Cons 👎 App内で複数VersionのModelをLoadしておくResourceが必要 ざっくり2種類の方法がある
  75. ©2023 CyberAgent Inc. Distribution prohibited Today’s Goal • Portableな実験環境 ◦

    Docker, Docker-Compose • Scalableな推論App ◦ App Runner まずは実験コードのDocker化から始めてみる 107