CircleCIの高速化🚀 / CircleCI faster

【オフライン開催】Omotesando.rb #92
https://omotesandorb.connpass.com/event/302869/

Example code to skip tests if source trees are the same.
https://gist.github.com/sinsoku/7b4201787d36f8ef669abd38395a28db

Transcript

1. CircleCIͷߴ଎Խ Omotesando.rb #92 2023/12/07 1
2. ࣗݾ঺հ • ໊લ: ਖ਼ಙ ޼(aka: ਆ଎) • GitHub: @sinsoku (ը૾ӈ্) • Twitter: @sinsoku_listy (ը૾ӈԼ) • ޷͖ͳݴޠ: Rust ! , Ruby " • Railsྺ: 8೥͘Β͍ 2
3. ࿩͢͜ͱ • ! CIΛ଎͘͢ΔΞΠσΞ • " CircleCIͷ࢓༷ͱ੍໿ • ⭐ ίʔυ঺հ ! Rubyͷ࿩͸͋Γ·ͤΜ 3
4. ! CIΛ଎͘͢ΔΞΠσΞ ιʔείʔυͷมߋ͕ͳ͍৔߹ɺςετΛ࣮ߦ͢Δඞཁ͕ͳ͍ɻ • fast-forward ͳίϛοτ • ίϛοτϝοηʔδ͚ͩมߋͨ͠ίϛοτ 4
5. ิ଍: fast-forward ӈਤͷϚʔδίϛοτ͸ࠨਤBͷιʔείʔυͱಉ͡ɻ 5
6. ྫ: rails/rails ࣮ࡍʹ rails/rails ͷϩάͰௐ΂ͯΈΔɻ ҎԼͷྫͩͱ f915f412c5 ͱ 64f57c2c9d ͕ಉ͡ιʔείʔυͰ ͢ɻ $ git log --oneline --graph -n 3 * f915f412c5 (HEAD -> main) Merge pull request #49701 from eileencodes/use-custom-class-for-pending-migrations-connection |\ | * 64f57c2c9d Use custom class for pending migrations connection |/ * cae5bafbf2 Merge pull request #49700 from akhilgkrishnan/active-job-argument-typo |\ 6
7. Tree object ͸ͲͪΒ΋ 0d51392 $ git cat-file -p f915f412c5 tree 0d513921cca36506f23afaaea4f252ca4563a1ab parent cae5bafbf2e74dba245e5afc0279a0aafd1ae156 parent 64f57c2c9d7b356ebd120751171a0fadf8946e90 author Eileen M. Uchitelle <eileencodes@users.noreply.github.com> 1697735828 -0400 committer GitHub <noreply@github.com> 1697735828 -0400 Merge pull request #49701 from eileencodes/use-custom-class-for-pending-migrations-connection $ git cat-file -p 64f57c2c9d tree 0d513921cca36506f23afaaea4f252ca4563a1ab parent cae5bafbf2e74dba245e5afc0279a0aafd1ae156 author eileencodes <eileencodes@gmail.com> 1697733331 -0400 committer eileencodes <eileencodes@gmail.com> 1697733713 -0400 Use custom class for pending migrations connection 7
8. طʹଞͷํ΋ϒϩάʹॻ͍͍ͯΔɻ1 1 https://songmu.jp/riji/entry/2021-03-08-utilize-git-tree-hash-in-testing.html 8
9. ͜ͷϒϩάͷख๏ 1. ςετ͕௨ͬͨπϦʔͷϋογϡ஋Λه࿥͢Δ • S3ʹϋογϡ஋ͷΦϒδΣΫτΛPUT 2. ςετ࣮ߦલʹS3ΦϒδΣΫτΛGET • Ωʔ͕ଘࡏ͢Ε͹ςετΛεΩοϓ 9
10. ϒϩά͔ΒҾ༻ - run: name: go test command: | repotree_hash=$(git rev-parse HEAD^{tree}) set +e curl -f https://${treehash_bucket}.s3.amazonaws.com/${repotree_hash} 2>/dev/null status=$? set -e if [ $status = "0" ]; then echo 'skip testing because the test for the same source tree has already passed' exit 0 fi go test ./... 10
11. ͜ͷख๏ͷ՝୊ • Amazon S3͕ඞཁ • γΣϧεΫϦϓτͰεΩοϓ൑அΛ͍ͯ͠Δ • δϣϒ୯ҐͰεΩοϓͰ͖ͳ͍ ͜ΕΒΛվળͰ͖ͳ͍͔ௐ΂ͨ 11
12. ! CircleCIͷ࢓༷ͱ੍໿ .circleci/config.yml ͷ࢓༷͸͜͜ʹશͯॻ͔Ε͍ͯΔɻ Configuration reference https://circleci.com/docs/configuration-reference/ 12
13. ࢖͑ͦ͏ͳػೳΛ୳͢ 13
14. The when Step 2 jobs: # conditional steps may also be defined in `commands:` job_with_optional_custom_checkout: parameters: custom_checkout: type: string default: "" machine: image: ubuntu-2004:202107-02 steps: - when: condition: <<parameters.custom_checkout>> steps: - run: echo "my custom checkout" - unless: condition: <<parameters.custom_checkout>> steps: - checkout 2 https://circleci.com/docs/configuration-reference/#the-when-step 14
15. ͜Μͳײ͡Ͱ͍͚ͦ͏ʁ jobs: test: parameters: previous_state: type: string default: "" machine: image: ubuntu-2004:202107-02 steps: - unless: condition: equal: [ success, << parameters.previous_state >> ] steps: - run: echo "Run Tests" - when: condition: equal: [ success, << parameters.previous_state >> ] steps: - run: echo "Skip Tests" 15
16. The when Step ͷݶք 16
17. ࣄલδϣϒʢεςοϓʣͷ஋͸࢖͑ͳ͍ 3 3 GitHub Actions ͳΒ؆୯ʹ࣮ݱͰ͖Δ 17
18. Dynamic Configuration4 version: 2.1 setup: true orbs: continuation: circleci/continuation@0.1.2 jobs: setup: executor: continuation/default steps: - checkout - run: ./generate-config > generated_config.yml - continuation/continue: configuration_path: generated_config.yml workflows: setup: jobs: - setup • CircleCI ͷ ҋ ศརͳػೳ • ੜ੒ͨ͠YAMLͰCIΛ࣮ߦ • ྫͰ͸ ./generate-config εΫϦϓτͰੜ੒ 4 https://circleci.com/docs/using-dynamic-configuration/ 18
19. ͳΜͱ͔ͳΓͦ͏ 19
20. ⭐ ίʔυ঺հ # .circleci/config.yml version: 2.1 setup: true parameters: tree_sha_path: type: string default: ".git_tree_sha" tree_status_path: type: string default: ".gh_tree_status.json" orbs: continuation: circleci/continuation@1.0.0 gh: circleci/github-cli@2.2.0 commands: 20
21. save_previous_tree_status # ਌ίϛοτͷίϛοτεςʔλεΛGitHubͰऔಘͯ͠JSONʹอଘ͢Δɻ # ਌ίϛοτͷ tree sha ΛΩʔʹΩϟογϡʹอଘ͢Δɻ save_previous_tree_status: steps: - run: name: Save previous tree sha command: git rev-parse @^^{tree} > << pipeline.parameters.tree_sha_path >> # An access token with `statuses:read` permission must be set in the GITHUB_TOKEN environment variable. - run: name: Save previous commit status as tree status command: gh api "/repos/{owner}/{repo}/commits/$(git rev-parse @^)/status" > << pipeline.parameters.tree_status_path >> - save_cache: key: tree_status_{{ checksum "<< pipeline.parameters.tree_sha_path >>" }} paths: - << pipeline.parameters.tree_status_path >> - run: name: Remove previous files command: rm << pipeline.parameters.tree_sha_path >> << pipeline.parameters.tree_status_path >> 21
22. restore_current_tree_status # ݱίϛοτͷ tree sha ΛΩʔʹΩϟογϡΛ෮ݩ͢Δɻ restore_current_tree_status: steps: - run: name: Save current tree sha command: git rev-parse @^{tree} > << pipeline.parameters.tree_sha_path >> - restore_cache: key: tree_status_{{ checksum "<< pipeline.parameters.tree_sha_path >>" }} - run: name: Put empty file if tree status does not exist command: | if [ ! -e "<< pipeline.parameters.tree_status_path >>" ]; then touch << pipeline.parameters.tree_status_path >> fi 22
23. jobs: setup: executor: continuation/default steps: - checkout - gh/install - save_previous_tree_status - restore_current_tree_status - run: name: Embed previous status command: | PREVIOUS_STATE=$(cat << pipeline.parameters.tree_status_path >> | \ jq -r '.statuses[] | select(.context == "ci/circleci: test") | .state') # .circleci/continue_config.yml ͷ `_PREVIOUS_STATE` Λஔ׵ sed -i "s/_PREVIOUS_STATE/$PREVIOUS_STATE/" .circleci/continue_config.yml - run: cat .circleci/continue_config.yml - continuation/continue: configuration_path: .circleci/continue_config.yml workflows: setup: jobs: - setup 23
24. ϝΠϯͷYAML # .circleci/continue_config.yml version: 2.1 parameters: previous_state: type: string default: "_PREVIOUS_STATE" jobs: test: docker: - image: cimg/base:stable steps: - checkout - unless: condition: equal: [ success, << pipeline.parameters.previous_state >> ] steps: - run: echo "Run tests" - when: condition: equal: [ success, << pipeline.parameters.previous_state >> ] steps: - run: echo "Skip tests" workflows: main: jobs: - test 24
25. ! ಈ࡞֬ೝ 25
26. git commit --allow-empty Ͱςετ͕εΩοϓ 26
27. ࣮ࡍͷYAML͸gistࢀর5 5 https://gist.github.com/sinsoku/7b4201787d36f8ef669abd38395a28db 27
28. ! ͦͯ͠YAML৬ਓʹ... 28