Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CircleCIでtiming dataに基づいたテスト分割をDartで利用できるよう...

operandoOS
March 15, 2022

CircleCIでtiming dataに基づいたテスト分割をDartで利用できるようにした話.

CircleCIでtiming dataに基づいたテスト分割をDartで利用できるようにした話.

文字起こし
https://github.com/operando/Notes/tree/master/%E9%96%8B%E7%99%BA%C3%97%E3%83%86%E3%82%B9%E3%83%88%20LT%E4%BC%9A%20-%20vol.2%20%23devtestlt

CircleCIでtiming dataに基づいたテスト分割をDartで利用できるようにするサンプルプロジェクト
https://github.com/operando/dart-test-CircleCI-timing-data

開発×テスト LT会 - vol.2 #devtestlt
https://rakus.connpass.com/event/237602/

operandoOS

March 15, 2022
Tweet

More Decks by operandoOS

Other Decks in Technology

Transcript

  1. ࣗݾ঺հ • Shinobu Okano • @operandoOS / shinobu.dart • 10X,

    Inc. / Software Engineer • Flutter / Dart / Android / Automation
  2. timing dataΛར༻ͯ͠ 
 ςετ෼ׂ࣮ͯ͠ߦ͢Δྫʢrspecʣ circleci tests glob "test/**/*.rb" | circleci

    tests split --split-by=timings > /tmp/tests-to-run bundle exec rspec $(cat /tmp/tests-to-run)
  3. • circleci tests splitͰςετϑΝΠϧͷҰཡऔಘ • --split-by=timingsͰtiming dataΛར༻࣮͠ߦ؀ڥ͝ͱʹ 
 ςετ෼ׂΛߦ͏Α͏ʹࢦఆ •

    bundle exec rspec΁࣮ߦ؀ڥ͝ͱʹ෼ׂ͞ΕͨςετΛ 
 ࣮ߦ͢ΔΑ͏ʹࢦఆ timing dataΛར༻ͯ͠ 
 ςετ෼ׂ࣮ͯ͠ߦ͢Δྫʢrspecʣ
  4. Why Dart test? • ࢲ͕ॴଐ͢Δձࣾ 10XͰ࡞͍ͬͯΔαʔϏε StailerͰ͸ 
 API࣮૷ʹαʔόʔαΠυDartΛ࢖༻ •

    αʔόʔαΠυDartͰ͸CIʹCircleCIΛར༻͍ͯ͠Δ • DartͰςετΛॻ͘৔߹ʮtestʯύοέʔδ͕ϝδϟʔ • https://pub.dev/packages/test
  5. Dart sharding tests • testύοέʔδʹ͸ςετ෼ׂ͢Δػೳ͕͋Δʂʂ • ݱࡏ 10XͰ͸testͷػೳΛ࢖ͬͯCircleCI্ͰςετΛ෼ׂɾฒྻ࣮ߦͯ͠ ͍Δ •

    ͔͠͠ɺ͜ͷ෼ׂํ๏͸֤ςετέʔεͷ࣮ߦ࣌ؒͳͲΛݩʹ 
 ςετ෼ׂ͞ΕΔΘ͚Ͱ͸ͳ͍ • ͦͷͨΊExecutor͝ͱʹ࣮ߦ࣌ؒͷ͹Β͖͕ͭେ͖͍ͷ͕՝୊
  6. Dart sharding tests dart test --total-shards 3 --shard-index 0 path/to/test.dart

    dart test --total-shards 3 --shard-index 1 path/to/test.dart dart test --total-shards 3 --shard-index 2 path/to/test.dart
  7. Dart testͷ݁ՌΛJunit XMLͰग़ྗ͢Δ • Dart test͸ςετ݁ՌΛJSONग़ྗ͢Δػೳ͕͋Δ • ͦͷJSONग़ྗ͞Εͨ݁ՌΛJunit XMLʹม׵͢Δπʔϧ͕͋Δ •

    https://github.com/TOPdesk/dart-junitreport • ͜ΕΛ࢖ͬͯDart testͷςετ݁ՌΛJunit XMLͰग़ྗ͢Δ
  8. ςετ࣮ߦͷίϚϯυΛॻ͖׵͑Δ • timing dataΛར༻ͯ͠ςετ෼ׂΛߦ͍ɺ෼ׂͨ͠ϑΝΠϧ ͷΈͰςετ࣮ߦ͢ΔΑ͏ʹίϚϯυΛॻ͖׵͑Δ TESTFILES=$(circleci tests glob "test/**/*_test.dart" |

    circleci tests split --split-by=timings) dart run test ${TESTFILES} --file-reporter json:test_results/tests_report.json --reporter expanded
  9. Կ͕ى͖͔ͨ...😨 • "No timing found for <ϑΝΠϧ໊>"ͱ͍͏ϝοηʔδ͕ग़ͨ • ςετϑΝΠϧʹର͢Δtiming data͕ଘࡏ͠ͳ͍ͥʂͬͯݴͬͯΔ

    • timing dataͷϑΝΠϧ͸ੜ੒͞Ε͍͕ͯͨɺԿ͔͠ΒͷݪҼͰ 
 ͏·͘ར༻͞Ε͍ͯͳ͍ͷͰɺ࠷దͳςετ෼ׂ͕ߦ͍͑ͯͳ͍
  10. <testsuites> <testsuite errors="0" failures="0" tests="2" skipped="0" name=“test.sample_1" > <properties> <property

    name="platform" value="vm" /> </properties> <testcase classname="test.sample_1" name=“String…” time="0.027" /> <testcase classname="test.sample_1" name=“String.trim…” time="0.005" /> </testsuite> <testsuite errors="0" failures="0" tests="4" skipped="0" name="test.sample_2" > <properties> <property name="platform" value="vm" /> </properties> <testcase classname="test.sample_2" name="String .s…” time="0.027" /> </testsuite> </testsuites> ग़ྗ͞ΕͨJunit XMLͷ಺༰
  11. version: 2.1 jobs: test: docker: - image: dart:2.16.1 parallelism: 2

    steps: - checkout - run: name: Download dependencies command: pub get - run: name: Test command: | TESTFILES=$(circleci tests glob "test/**/*_test.dart" | circleci tests split --split-by=timings) dart run test ${TESTFILES} --file-reporter json:test_results/tests_report.json --reporter expanded - run: name: Install JUnitReport when: always command: dart pub global activate --source git https://github.com/operando/dart-junitreport.git - run: name: Convert tests to JUnit when: always command: | $HOME/.pub-cache/bin/tojunit --input test_results/tests_report.json --output test_results/tests_report_junit.xml sed -i -e "s|file:///root/project/||g" test_results/tests_report_junit.xml - store_test_results: path: test_results/ - store_artifacts: path: test_results/ workflows: version: 2 build: jobs: - test .circleci/con fi g.ymlͷ಺༰ 
 ௕͍….จࣈখ͍͞…😇
  12. - run: name: Test command: | TESTFILES=$(circleci tests glob "test/**/*_test.dart"

    | circleci tests split --split-by=timings) dart run test ${TESTFILES} --file-reporter json:test_results/tests_report.json --reporter expanded - run: name: Install JUnitReport when: always command: dart pub global activate --source git https://github.com/operando/dart-junitreport.git - run: name: Convert tests to JUnit when: always command: | $HOME/.pub-cache/bin/tojunit --input test_results/tests_report.json --output test_results/tests_report_junit.xml sed -i -e "s|file:///root/project/||g" test_results/tests_report_junit.xml - store_test_results: path: test_results/ - store_artifacts: path: test_results/ େࣄͳͷ͸͜ͷ෦෼ʂ
  13. - run: name: Test command: | TESTFILES=$(circleci tests glob "test/**/*_test.dart"

    | circleci tests split --split-by=timings) dart run test ${TESTFILES} --file-reporter json:test_results/tests_report.json --reporter expanded - run: name: Install JUnitReport when: always command: dart pub global activate --source git https://github.com/operando/dart-junitreport.git - run: name: Convert tests to JUnit when: always command: | $HOME/.pub-cache/bin/tojunit --input test_results/tests_report.json --output test_results/tests_report_junit.xml sed -i -e "s|file:///root/project/||g" test_results/tests_report_junit.xml - store_test_results: path: test_results/ - store_artifacts: path: test_results/ forkͨ͠ϦϙδτϦʹमਖ਼ΛೖΕͨͷͰ 
 ࣗ਎ͷϦϙδτϦ͔Βऔಘ
  14. - run: name: Test command: | TESTFILES=$(circleci tests glob "test/**/*_test.dart"

    | circleci tests split --split-by=timings) dart run test ${TESTFILES} --file-reporter json:test_results/tests_report.json --reporter expanded - run: name: Install JUnitReport when: always command: dart pub global activate --source git https://github.com/operando/dart-junitreport.git - run: name: Convert tests to JUnit when: always command: | $HOME/.pub-cache/bin/tojunit --input test_results/tests_report.json --output test_results/tests_report_junit.xml sed -i -e "s|file:///root/project/||g" test_results/tests_report_junit.xml - store_test_results: path: test_results/ - store_artifacts: path: test_results/ Junit XMLͰग़ྗ͠ɺsedͰ fi le attributeͷ಺༰Λॻ͖׵͑
  15. - run: name: Test command: | TESTFILES=$(circleci tests glob "test/**/*_test.dart"

    | circleci tests split --split-by=timings) dart run test ${TESTFILES} --file-reporter json:test_results/tests_report.json --reporter expanded - run: name: Install JUnitReport when: always command: dart pub global activate --source git https://github.com/operando/dart-junitreport.git - run: name: Convert tests to JUnit when: always command: | $HOME/.pub-cache/bin/tojunit --input test_results/tests_report.json --output test_results/tests_report_junit.xml sed -i -e "s|file:///root/project/||g" test_results/tests_report_junit.xml - store_test_results: path: test_results/ - store_artifacts: path: test_results/ ग़ྗͨ͠Junit XMLΛอଘʂ
  16. - run: name: Test command: | TESTFILES=$(circleci tests glob "test/**/*_test.dart"

    | circleci tests split --split-by=timings) dart run test ${TESTFILES} --file-reporter json:test_results/tests_report.json --reporter expanded - run: name: Install JUnitReport when: always command: dart pub global activate --source git https://github.com/operando/dart-junitreport.git - run: name: Convert tests to JUnit when: always command: | $HOME/.pub-cache/bin/tojunit --input test_results/tests_report.json --output test_results/tests_report_junit.xml sed -i -e "s|file:///root/project/||g" test_results/tests_report_junit.xml - store_test_results: path: test_results/ - store_artifacts: path: test_results/ ͜ΕͰ͜ͷ෦෼ͷςετ෼ׂ͕͏·͘ಈ͍ͨʂ