20240920_component_testing.pdf

Transcript

1. ্खʹ޲͖߹͏ ίϯϙʔωϯτςετ 2024.09.20 @Quramy
2. About me - id: @Quramy (GitHub / X ) - ͓࢓ࣄ: Web ϑϩϯτΤϯυΤϯδχΞ - React, Next.js, GraphQL - ՝֎׆ಈ: ςετࢧԉܥπʔϧͷ࡞੒ - Prisma ORM ؔ࿈: jest-prisma, prisma-fabbrica - Visual Testing ؔ࿈: reg-suit, Storycap
3. Visual Testing ಉ޷ձ https://speakerdeck.com/quramy/reg-viz-vrt-tools?slide=6
4. Agenda - ίϯϙʔωϯτςετͷಋೖ - ίϯϙʔωϯτςετͷӡ༻
5. ಋೖฤ
6. ࣗಈςετͷ໨త - εϐʔυΛଛͳΘͣʹ։ൃΛଓ͚ΔͨΊ - طଘͷػೳ͕໰୊ͳ͘ಈ࡞͚͍ͭͮͯ͠Δ͜ͱΛ୲อͯ͘͠ΕΔ - ϑϩϯτΤϯυͷ৔߹ʮݟͨ໨͕ͲͷΑ͏ʹมߋ͞ΕΔͷ͔ʯ͕Մࢹ Խ͞ΕΔͱɺϨϏϡΞଆͷ֬ೝίετ࡟ݮʹ΋ͭͳ͕Δ
7. උ͑͸େࣄ - ϓϩδΣΫτ্ཱͪ͛࣌͸ίϯϙʔωϯτςετಋೖΛݟૹΔͷ΋બ୒ ࢶͱͯ͠͸ΞϦͱࢥ͏ - ͨͩ͠ɺޙ͔ΒಋೖͰ͖ΔΑ͏ʹඋ͓͑ͯ͘͜ͱ͸ॏཁ - ϓϩδΣΫτ͕ҭ͔ͬͯΒಥવʮΑ͠ ίϯϙʔωϯτͷ Visual Testing ΍Ζ͏ʂʯͱٸʹࢥཱ͍ͬͯ΋ɺθϩ͔ΒςετέʔεΛॻ͘ͷ͸େม
8. ίϯϙʔωϯτςετͱ Storybook ίϯϙʔωϯτςετʹ Storybook Λ༻͍͍ͯΔཧ༝ - UI ίϯϙʔωϯτ։ൃج൫ͱͯ͠Չ઎ঢ়ଶ (ݸਓతʹ͸ଞͷπʔϧʹ΋ؤுͬͯ΄͍͕͠ɺStorybook ʹউͭͷ͕૬౰େมͳঢ়گʹͳ͍ͬͯ Δͱࢥ͏) - Story ͕ςετέʔεͱͯ͠ྲྀ༻ՄೳͰ͋ΔͨΊɺʮͱΓ͋͑ͣ Story ͚ͩॻ ͘Α͏ʹ͓ͯ͘͠ʯ͕ίϯϙʔωϯτςετͷඋ͑ͱͳΔ - Storybook ͕ࣗಈςετʹؔ͢ΔػೳڧԽʹྗΛೖΕ͍ͯΔ - Jest / Vitest ࿈ܞ (compositeStories), test-runner, Chromatic, etc...
9. ίϯϙʔωϯτςετ with Storybook Storybook ͱ࿈ܞ͢Δςετπʔϧ΍ςετϓϥοτϑΥʔϜ͸༷ʑͳ΋ͷ ͕͋Δɻओཁͳ΋ͷΛ঺հ (1/4) - Chromatic: https://www.chromatic.com/ Storybook ͷϗεςΟϯά΍ࣗಈςετΛ݉Ͷඋ͑ͨ all-in-one ͳαʔϏ εɻStorybook ΦϑΟγϟϧ - Lost Pixel: https://www.lost-pixel.com/ VRT πʔϧɻChromatic ͷΑ͏ͳ SaaS ͱͯ͠ͷఏڙܗଶͷଞɺࣗલͷ CI ʹηϧϑϗετ͢Δ͜ͱ΋Մೳͳπʔϧ
10. ίϯϙʔωϯτςετ with Storybook Storybook ͱ࿈ܞ͢Δςετπʔϧ΍ςετϓϥοτϑΥʔϜ͸༷ʑͳ΋ͷ ͕͋Δɻओཁͳ΋ͷΛ঺հ (2/4) - Storycap: https://github.com/reg-viz/storycap Storybook Λ Puppeteer ͰΫϩʔϧͯ͠ը૾Խ͢Δ CLI reg-suit ΍ reg-actions (͜ΕΒ΋ reg-vizͷπʔϧ) ͱ૊Έ߹ΘͤΔ͜ͱͰ ίϯϙʔωϯτ୯Ґͷ Visual Regression Test Λ࣮ݱ͢Δɻ Storybook x VRT ͱͯ͠͸࿝ฮͷ෦ྨ (2017 ೥͔Β͋Δ)
11. ίϯϙʔωϯτςετ with Storybook Storybook ͱ࿈ܞ͢Δςετπʔϧ΍ςετϓϥοτϑΥʔϜ͸༷ʑͳ΋ͷ ͕͋Δɻओཁͳ΋ͷΛ঺հ (3/4) - storybook/test-runner: https://storybook.js.org/docs/writing-tests/test-runner Storybook ެࣜͷςετϥϯφʔɻࣗલ CI Ͱ Story ͷϨϯμϦϯά΍ Play function Λ࣮ߦՄೳɻ jest-playwright Λ಺෦తʹར༻͍ͯ͠Δ - storycap-testrun: https://github.com/reg-viz/storycap-testrun storybook/test-runner Ͱར༻͢ΔεΫϦʔϯγϣοτϥΠϒϥϦɻ
12. ίϯϙʔωϯτςετ with Storybook Storybook ͱ࿈ܞ͢Δςετπʔϧ΍ςετϓϥοτϑΥʔϜ͸༷ʑͳ΋ͷ ͕͋Δɻओཁͳ΋ͷΛ঺հ (4/4) - Jest / Vitest: ୯ମςετͷπʔϧͰ͋Δ͕ɺStorybook ͕ఏڙ͢Δ composeStories ؔ਺ͱ૊Έ߹ΘͤΔ͜ͱͰ Story ͷςετ͕Մೳɻ https://storybook.js.org/docs/writing-tests/import-stories-in-tests/ stories-in-unit-tests
13. ͓͜ͱΘΓ - චऀ͕࣮ࡍʹӡ༻ͨ͜͠ͱͷ͋Δίϯϙʔωϯτςετͷߏ੒͸ҎԼ: - Storybook + Storycap + reg-suit - Storybook + Jest + compositeStories - Ҏ߱Ͱ঺հ͢Δӡ༻ Tips ͸ɺπʔϧͷબ୒ͱಠཱͨ͠಺༰ͱͳΔΑ͏ۃ ྗ৺͕͚͍ͯ·͕͢ɺ্هΛཹҙͯ͠΋Β͑Δͱॿ͔Γ·͢
14. ӡ༻ฤ
15. ӡ༻ฤ ࡟Εʂ ࣮ߦ࣌ؒ
16. Կ෼·Ͱ଱͑ΒΕΔʁ - Pull Request ࣌ͷ CI ࣮ߦ࣌ؒɺͲΕ͘Β͍·ͰզຫͰ͖·͔͢ʁ - ݸਓࠩ͸͋Δ͕ɺࣗ෼ͷ؍ଌൣғͰ͸ 5෼ลΓʹᮢ஋͕͋Γͦ͏ - ίϯϙʔωϯτςετͷ࣮ߦ͕࣌ؒ௕͘ͳΕ͹ͳΔ΄ͲɺνʔϜͷϔΠ τ͕ཷ·͍ͬͯ͘
17. (ࢀߟ) ࣗಈςετ࣮ߦ࣌ؒਪҠ https://qiita.com/Quramy/items/46d0b09ae4d8887b0941  ߦ
    ίϯϙʔωϯτ
     
    έʔε $*
    ࣮ߦ࣌ؒ
18. ฒྻ࣮ߦ (ϫʔΧʔϨϕϧ) - CI ͷ࣮ߦ؀ڥΛCPUΛ૿ڧͭͭ͠ɺ࣮ߦϒϥ΢β (worker) ਺Λ૿΍͢ - GitHub Actions Ͱ͋Ε͹ɺ Self Hosted Runner Ͱӡ༻͍ͯ͠Δਓ޲͚ - ίϯϙʔωϯτςετΛ࣮ߦ͢Δ CLI ͷΦϓγϣϯͰ worker ਺Λมߋ: - Storycap CLI: --parallel Φϓγϣϯ - storybook/test-runner CLI: --max-workers Φϓγϣϯ
19. ฒྻ࣮ߦ (δϣϒϨϕϧ) - CI ͷ࣮ߦ؀ڥ͝ͱฒྻԽ͢Δύλʔϯ - GitHub Actions Ͱ͋Ε͹ matrix ͳͲͰ ฒྻδϣϒ࣮ߦՄೳ ※ ίϯϙʔωϯτςετ͕૸Δ·Ͱͷεςοϓ (؀ڥࣗମͷηοτΞοϓ ΍ npm i ͢Δ෦෼ ͳͲ) ͕େ͖͍ͱɺฒྻԽޮ཰͕Լ͕ΔͨΊ޻෉͕ඞཁ - storybook/test-runner ΍ Storycap ͷ --shard ΦϓγϣϯΛར༻͢Δ
20. ϥϯφʔΛ࢖͍෼͚Δ - ࣮ϒϥ΢βΛ࢖ͬͨίϯϙʔωϯτςετ͸ͦ΋ͦ΋஗͍ - શͯͷίϯϙʔωϯτͷςετͰɺৗʹϒϥ΢β͕ඞཁͱ͍͏Θ͚Ͱ͸ ͳ͍͸ͣ (e.g. ϑΥʔϜόϦσʔγϣϯ΍ aria ଐੑͷ֬ೝ) - jsdom Ͱे෼ͳέʔεͰ͋Ε͹ɺJest / Vitest + compositeStories ͷςε τίʔυͱ࣮ͯ͠ߦͰ͖Ε͹ɺ஗͘ͳΓʹ͍͘ - Storycap (= Puppeteer): ਺100 msec / story - Jest + jsdom: ਺10 msec / story
21. ࣮ફฤ ௵ͤʂ Flakiness
22. Flaky Test μϝθολΠ - Flaky ͳςετͱ͸ - ιʔείʔυʹมߋΛՃ͍͑ͯͳ͍ͷʹ ੒ޭ/ࣦഊ ͕҆ఆ͠ͳ͍ঢ়ଶ - ِཅੑ (False Positive) ͸ࣗಈςετͷఢ - ಛʹ Visual Testing ͸ݕূʹը૾Λ༻͍ΔͨΊ Flaky ςετʹؕΓ΍͍͢
23. VRT ʹ͓͚Δᮢ஋ͷௐ੔ - ϒϥ΢βʹίϯϙʔωϯτΛඳըͤ͞Δ౎߹্ɺ׬શͳ pixel by pixel ͷ Ұக͸೉͍͠ - e.g. CSS ͷ border-radius ΍ box-shadow ΍ ΞϯνΤΠϦΞε - [IMO] 100 ϐΫηϧ (ॎԣ 10px ͷਖ਼ํܗͷ໘ੵ) ఔ౓ͷࠩҟ͸ڐ༰ - 100 ఔ౓ͰِӄੑʹͳΔ͜ͱ͸໓ଟʹͳ͍ - [IMO] ը૾શମʹର͢Δൺ཰Ͱࢦఆ͢ΔͷͰ͋Ε͹ɺ 0.01% Λج४ʹ
24. ෦෼తͳࠩ͠ସ͑ ίϯϙʔωϯτͷҰ෦͕ෆ҆ఆͳ৔߹ ( e.g. YouTube ຒࠐಈը, iframe) ɺ VRT ࣌ͷΈผ div ʹࠩ͠ସ͑ͯɺFlaky Test ʹͳΒͳ͍Α͏ʹ͢Δ export function MyComponent() { return ( <div> {/* ͕͜͜ෆ҆ఆͳཁૉ */} <iframe style={{ height: 400, width: "100%" }} src="https://example.com/hogehoge" /> </div> ); } import { isScreenshot } from "storycap"; export function MyComponent() { return ( <div> {process.env.NODE_ENV !== "production" && isScreenshot() ? ( <div style={{ height: 400, background: "#888" }} /> ) : ( <iframe style={{ height: 400, width: "100%" }} src="https://example.com/hogehoge" /> )} </div> ); }
25. ͦΕͰ΋μϝͳΒ skip - Ͳ͏ͯ͠΋ Flakiness Λ௵ͤͳ͍৔߹ɺͦͷέʔε(= Story) Λ ࣗಈςετର ৅֎ʹׂͯ͠Ε૭ԽΛ๷ࢭ͢Δ - ίʔυΛมߋͯ͠΋͍ͳ͍ͷʹ CI Ͱ VRT Λ૸ΒͤΔͨͼʹࠩ෼͕ग़ͯ͠· ͏ঢ়گΛ์ஔ͢ΔΑΓ͸Ϛγ export const MyStory = { args: {}, // storybook/test-runner ͷ৔߹ tags: ["skip"], parameters: { screenshot: { // Storycap ͷ৔߹ skip: true, }, }, } satisfies Story;
26. ͓ΘΓʹ - ίϯϙʔωϯτςετӡ༻ͷίπ - Slow Test / Flakey Test ͸ݟա͝͞ͳ͍ - Off topics - Ұൠతͳࣗಈςετϊ΢ϋ΢ (e.g. ςετίʔυͷೝ஌ෛՙରࡦ) ΋׆ ༻ͯ͠ɺΑΓշదͳϑϩϯτΤϯυςετΛ໨ࢦ͠·͠ΐ͏ - e.g. https://gihyo.jp/list/group/αόϯφศΓ- ～ ιϑτ΢ΣΞ։ൃͷ ߥ໺Λੜ͖ൈ͘ ～
27. Thank you !