機械学習チームのモノレポ移行

「モノレポへの移行 LT -生産性の高いアーキテクチャに向けた第一歩-」の発表資料

https://findy.connpass.com/event/296339/

Transcript

1. Koki Nishihara
2. Koki Nishihara (github ) CADDi AI Team, Tech Lead MLOps Complicated-subsystem team OSS pydantic, Pants 10 27 @nishikoh PyCon APAC 2023
3. LT CADDi Tech Blog
4. CI Pull Request
5. CI Pull Request
6. Bazel Pants : 3rd party plugin : Python Python Pants Bazel Pants
7. polyrepo monorepo + dir BUILD polyrepo monorepo app 1 ├── src 2 │ └── main.py 3 ├── Dockerfile 4 ├── poetry.lock 5 ├── pyproject.toml 6 └── tests 7 └── test_main.py 8 pants.toml 1 projects 2 ├── app_1 3 │ ├── src 4 │ │ ├── BUILD 5 │ │ └── main.py 6 │ ├── BUILD 7 │ ├── Dockerfile 8 │ ├── poetry.lock 9 │ ├── pyproject.toml 10 │ └── tests 11 │ ├── BUILD 12 │ └── test_main.py 13 └── app_N 14 ├── ... 15
8. pants.toml [GLOBAL] 1 pants_version = "2.17.0" 2 backend_packages = [ 3 "pants.backend.python", 4 "pants.backend.python.lint.black", 5 "pants.backend.python.lint.flake8", 6 "pants.backend.python.lint.isort", 7 "pants.backend.python.lint.docformatter", 8 "pants.backend.python.lint.bandit", 9 "pants.backend.python.lint.autoflake", 10 "pants.backend.python.lint.pyupgrade", 11 "pants.backend.python.lint.pylint", 12 "pants.backend.experimental.python.lint.ruff", 13 "pants.backend.experimental.python.lint.add_trailing_comma", 14 "pants.backend.python.typecheck.mypy", 15 ] 16 17
9. --changed-since --changed-dependees=transitive CI p50 30 , p75 90 , max 20 format lint test pants fmt --changed-since=origin/main --changed-dependees=transitive 1
10. Python CADDi Tech Blog
11. WebAPI Python
12. 3rd party package : 300 : 30 3rd party package 300 packege 5
13. Good CI/CD format 1
14. More must git clone git lfs .git/ GitHub Actions disk git lfs 300 3rd party
15. More Pants model registry git lfs GitHub Actions hosted runner Cloud Build GitHub Actions larger runners
16. CADDi Tech Blog
17. Software Engineering at Google Build Systems and Build Philosophy
18. PyCon APAC 2023 “Pants Python Monorepo” Python, Pants
19. CADDi https://recruit.caddi.tech