runnによるAPIのシナリオテストの導入と自動化 / stac2022

発表者はYAMLベースでAPIのシナリオテストを書くことが可能なrunn (https://github.com/k1LoW/runn) というツールを開発しています。本発表ではrunnの導入や事例、開発者の想定と実際について紹介します。

Transcript

1. runnʹΑΔAPIͷγφϦΦςε τͷಋೖͱࣗಈԽ খࢁ݈Ұ࿠ / GMO PEPABO inc. 2022.12.3 ιϑτ΢ΣΞςετࣗಈԽΧϯϑΝϨϯε2022 1
2. ࣗݾ঺հ ٕज़෦ɹٕज़ج൫νʔϜ 2018೥ த్ೖࣾ খࢁ ݈Ұ࿠ Ken’ichiro Oyama গ࣮͠༻తͰখ͞ͳOSSΛॻ͘ͷ͕झຯɻ • GitHub : k1LoW • Twitter : @k1LoW 2
3. GMOϖύϘʹ͍ͭͯ 3 https://pepabo.com
4. • https://github.com/k1LoW/awspec • AWSͷ֤Ϧιʔεͷঢ়ଶʹରͯ͠ɺRSpecͰςετΛॻ͘͜ͱ͕Ͱ͖Δπʔϧ • ݱࡏ88Ϧιʔεαϙʔτ͍ͯ͠Δ awspec 4 4 ࡞͍ͬͯΔOSS঺հ
5. 5 • https://github.com/k1LoW/tbls • CIϑϨϯυϦͳσʔλϕʔευΩϡϝϯτੜ ੒πʔϧ • PostgreSQL, MySQLͳͲͷRDBMS͚ͩͰͳ ͘ɺBigQuery΍Amazon DynamoDB, Cloud Spannerͱ͍ͬͨσʔλιʔεʹ΋ରԠ • υΩϡϝϯτΛॆ࣮ͤ͞Δػೳͱͯ͠ϝλ σʔλΛΞϊςʔγϣϯ͢Δ͜ͱ͕Մೳ • ςʔϒϧɺΧϥϜͳͲͷίϝϯτ෇༩͚ͩͰ ͳ͘λάɺϦϨʔγϣϯͳͲ tbls ࡞͍ͬͯΔOSS঺հ
6. • https://github.com/k1LoW/octocov • ίʔυϝτϦΫεπʔϧ…ίʔυΧόϨοδɺCode to Test Ratioɺςετ࣮ߦ࣌ؒ • ଟ਺ͷϓϩάϥϛϯάݴޠͱɺओཁͳΧόϨοδϨϙʔτϑΥʔϚοτʹରԠ • GitHub Actionsͷactionͱͯ͠ίʔυϝτϦΫεΛूܭɺP/RίϝϯτɺϨϙʔτ • CLIπʔϧͱͯ͠ίʔυΧόϨοδΛूܭɻදࣔ • ΧόϨοδόοδੜ੒ octocov 6 6 ࡞͍ͬͯΔOSS঺հ
7. ΞδΣϯμ • runnͱ͸ • ςετπʔϧ։ൃऀͱͯ͠ͷ૝ఆϢʔεέʔε • ࣮ࡍͷϢʔεέʔε • ͓ΘΓʹ ɹ※ͳ͓ɺຊൃද͸Θ͔Γ΍͘͢͢ΔͨΊʹ࣮ࡍͷ࣌ܥྻʹԊ͍ͬͯͳ͍಺༰ʹͳ͍ͬͯ·͢ɻ 7
8. 8 runnͱ͸
10. • runn ( means "Run N". is pronounced /rʌ́n én/. ) is a package/tool for running operations following a scenario. • γφϦΦΛYAMLͰॻ͍ͯͦΕΛ΋ͱʹૢ࡞ʢΦϖϨʔγϣϯʣΛࣗಈԽͰ͖Δύο έʔδ/πʔϧ • Goͷύοέʔδͱͯ͠΋ίϚϯυʢrunnίϚϯυʣͱͯ͠΋࢖͑Δ • runn run [γφϦΦσΟϨΫτϦ]/*.yml • ʮΦϖϨʔγϣϯࣗಈԽʯͱ޿ൣғΛλʔήοτʹ͍ͯ͠Δ͕ɺAPIͷγφϦΦς ετ͕ϑΝʔετϓϥΠΦϦςΟͳϢʔεέʔεʢ։ൃͷ͖͔͚ͬͰ΋͋ΔͨΊʣ 10 https://github.com/k1LoW/runn runn
11. • 1όΠφϦͷγφϦΦ࣮ߦͷCLIίϚϯυʢπʔϧʣͱͯ͠࢖༻Ͱ͖Δ • 1όΠφϦ = CIϑϨϯυϦ • ͞·͟·ͳϓϩτίϧʹରԠ͓ͯ͠Γɺ1ͭͷγφϦΦ಺Ͱෳ਺૊Έ߹ΘͤΔ͜ͱ͕Մೳ • HTTP • gRPC • DatabaseʢSQLʣ • Chrome DevTools Protocol • ೚ҙίϚϯυͷ࣮ߦʢϩʔΧϧ/SSHʣ • GoͷTest Helperʢύοέʔδʣͱͯ͠࢖༻Ͱ͖Δ 11 https://github.com/k1LoW/runn runnͷಛ௃ʢv0.53.0ʣʢ1/2ʣ
12. 12 https://github.com/k1LoW/runn runnͷಛ௃ʢv0.53.0ʣʢ2/2ʣ • γφϦΦϑΝΠϧʢYAMLʣͷϑΥʔϚοτ͕OpenAPI SpecϥΠΫ • γφϦΦ಺ͷεςοϓؒͷ஋͕ࣗಈͰ࿈ܞ͞Ε͍ͯΔ • ʮεςοϓAΛ࣮ߦ͔ͯ͠Βͦͷ݁ՌΛ࢖ͬͯεςοϓBΛ࣮ߦ͢Δʯͱ͍͏γφϦΦΛߏங͠΍͍͢ • ෳ਺ͷγφϦΦͷ࣮ߦΛ૝ఆ࣮ͨ͠૷ • ෳ਺ͷγφϦΦͷ෼ׂ࣮ߦɺαϯϓϦϯά࣮ߦɺϥϯμϜ࣮ߦͳͲ͕Մೳ • ϧʔϓ࣮ߦ΍ϦτϥΠ࣮ߦͷ࢓૊Έ͕૊ΈࠐΈͰඋΘ͍ͬͯΔ • ಉ͡γφϦΦϑΝΠϧΛ࢖ͬͨෛՙςετʢϩʔυςετʣ͕Մೳ • ίϚϯυΛ runn run ͔Β runn loadt ʹมߋ͢Δ͚ͩɻෳ਺γφϦΦ࣮ߦ΋ͦͷ··Մೳ • γφϦΦ࣮ߦͷϩά΋ه࿥Մೳ • γφϦΦ࣮ߦͷϓϩϑΝΠϧ΋औಘՄೳʢ֤εςοϓͷܦա࣌ؒʣ
13. • runn͸1ͭҎ্ͷγφϦΦΛ࣮ߦͯ͠ɺͦΕͧΕͷ੒ޭ/ࣦഊΛΈΔ • γφϦΦ͸1ͭҎ্ͷεςοϓ͔Βߏ੒͞Ε͍ͯΔ • 1γφϦΦ1YAMLϑΝΠϧʹͳ͍ͬͯΔʢRunbookͱݺΜͰ͍Δʣ • γφϦΦ͝ͱʹRunnerͱݺΜͰ͍Δεςοϓ࣮ߦίϯϙʔωϯτΛఆٛ͠ɺͦͷRunnerΛ࢖֤ͬͯ εςοϓΛ࣮ߦ͍ͯ͘͠ • Runnerʹ͸γφϦΦ಺ʹఆ͕ٛඞཁͳ௨ৗͷRunnerͱɺطʹrunnͰ༻ҙ͍ͯ͠ΔBuilt-in runnerͷ2 छྨ͕͋Δ • Runner: HTTP Runner, DB Runner, gRPC Runner, CDP Runner, SSH Runner • Built-in runner: Exec runner, Test runner, Dump runner, Include runner, Bind runner 13 https://github.com/k1LoW/runn runnͷΞʔΩςΫνϟ
14. 14 γφϦΦʢRunbookʣΛॻ͘
15. 15 https://github.com/k1LoW/runn
16. 16 https://github.com/k1LoW/runn
17. 17 https://github.com/k1LoW/runn
18. 18 https://github.com/k1LoW/runn
19. 19 https://github.com/k1LoW/runn
20. 20 https://github.com/k1LoW/runn
21. 21 https://github.com/k1LoW/runn
22. 22 https://github.com/k1LoW/runn Runnerͷઃఆ ม਺ͷઃఆ ɹɹɹɹ֤εςοϓ 0 1 2 3
23. 23 runn new
24. • γφϦΦϑΝΠϧ࡞੒ͷͨΊͷαϙʔτίϚϯυ • runn newͷҾ਺ʹ஋Λ౉͢͜ͱͰγφϦΦϑΝΠϧΛ࡞੒͢Δ • cURLίϚϯυ • gRPCurlίϚϯυ • ΞΫηεϩά • ʢOpenAPI SpecͷυΩϡϝϯτʁʣ • ʢPostmanͷCollectionʁʣ • طଘͷγφϦΦʹεςοϓͱͯ͠௥Ճ͢Δ͜ͱ΋Մೳ 24 https://github.com/k1LoW/runn runn new
25. 25 https://github.com/k1LoW/runn DEMO: https://github.com/k1LoW/runn#quickstart
26. 26 ςετπʔϧ։ൃऀͱͯ͠ͷ ૝ఆϢʔεέʔε
27. • ൃදऀ͸ιϑτ΢ΣΞΤϯδχΞ • ओʹόοΫΤϯυͷઃܭͱ։ൃΛ୲౰ • ςετΛઐ໳ʹ͍ͯ͠ΔΤϯδχΞͰ͸ͳ͍ • GoͰAPIαʔόΛ։ൃ͍ͯ͠Δ • ୲౰ίϯϙʔωϯτʢAPIαʔόʣΛઃܭ͍ͯ͘͠ʹ͋ͨͬͯɺෳ਺ͷAPIΛ·͍ͨ Ͱ1ͭͷػೳΛ࣮ݱ͢ΔΑ͏ͳઃܭʹͳΔ͜ͱ͕ݟ͖͑ͯͨɻ • APIεΩʔϚͱσʔλϕʔεεΩʔϚ͸৻ॏʹઃܭͨ͠ɻ͔͠͠ɺίʔυϕʔε͸࠷ ॳ͔Β࠷దͳΞʔΩςΫνϟΛબ୒Ͱ͖Δͱ͸ࢥ͑ͳ͔ͬͨɻ 27 ςετπʔϧ։ൃऀͱͯ͠ͷ૝ఆϢʔεέʔε ։ൃͷಈػ
28. • ػೳ୯Ґʢෳ਺APIάϧʔϓ͔ΒͳΔ͠͹͠͹ εςʔτϑϧͳػೳʣͷςετΛॆ࣮ͤ͞Δ͜ ͱʹΑΓɺͨͱ్͑தͷϦΞʔΩςΫςΟϯά ͕͋ͬͯ΋ػೳͷอূ͕Ͱ͖ΔΑ͏ʹ͍ͨ͠ɻ • ։ൃॳظ͔ΒUnitςετͱಉ༷ʹػೳʹରͯ͠ ͷγφϦΦςετͷॆ࣮Λ໨ࢦ͢ɻ • ͦͷͨΊʹγφϦΦςετΛUnitςετฒΈʹ ؆୯ʹ͢Δπʔϧͱͯ͠runnΛ։ൃɻ 28 ςετπʔϧ։ൃऀͱͯ͠ͷ૝ఆϢʔεέʔε Testing trophyʢςεςΟϯάτϩϑΟʔʣ https://kentcdodds.com/blog/the-testing-trophy-and-testing-classifications 
 https://twitter.com/kentcdodds/status/960723172591992832
29. • GoͰAPIαʔόΛ։ൃ͍ͯ͠Διϑτ΢ΣΞΤϯδχΞ͕ςετίʔυʹ૊Έ͜ΜͰ࢖ ༻͢Δ • net/http/httptest ύοέʔδͱ૬ੑ͕͍͍ • γφϦΦͷ֦ॆ΋YAMLΛॻ͚ͩ͘Ͱྑ͍ • OpenAPI SpecϥΠΫͳϑΥʔϚοτΛ࠾༻ͨ͠ͷ΋ɺγφϦΦςετΛॻ͘Τϯ δχΞͱOpenAPIυΩϡϝϯτΛॻ͘ΤϯδχΞΛಉҰਓ෺ͱͯ͠ߟ͍͑ͯͨͨΊ • 1όΠφϦͷͨΊΠϯετʔϧ΋؆୯ɻCI΁ͷ૊ΈࠐΈ΋༰қɻ 29 ςετπʔϧ։ൃऀͱͯ͠ͷ૝ఆϢʔεέʔε ։ൃऀ͕૝ఆ͍ͯͨ͠ओͳϢʔεέʔε
30. ࣮ࡍͷϢʔεέʔε 30
31. • ݁࿦ͱͯ͠͸ʮ͏·͍ͬͨ͘ʯ • Ұ౓૊ΈࠐΜͰ͠·͑͹ɺ͋ͱ͸YAMLΛॻ͚ͩ͘ʹͳΔͨΊɺ͔ͳΓḿͬͨɻ • ḿΓ͗ͯ͢ςετ࣮ߦ࣌ؒʹ૿େ͠ɺςετ෼ׂ͕ඞཁʹͳͬͨ + ςετ෼ׂػೳ΍ϓϩϑΝΠϧػೳͷ௥Ճ 31 ࣮ࡍͷϢʔεέʔε ։ൃऀ͕૝ఆ͍ͯͨ͠ϢʔεέʔεͰ͸͏·͍͔ͬͨ͘ʁ
32. 32 ʢ౰࣌ʣ ૝ఆ֎ͷϢʔεέʔε
33. • γφϦΦ͕OpenAPI SpecϥΠΫͰ͋Δ͜ͱ͔ΒΤϯδχΞʹͱͬͯಡΈ΍͘͢ɺͦ ͷ··ΦϯϘʔσΟϯάʹ໾ʹཱͬͨʢAPIར༻ଆ΁ͷઆ໌ͳͲʣ • ॻ͖΍͍͢=ಡΈ΍͍͢ + ΑΓυΩϡϝϯτͱͯ͠࢖͍΍͘͢͢ΔͨΊʹγφϦΦ΍εςοϓͷઆ໌Λهड़͢ ΔͨΊͷ desc: ηΫγϣϯͷ৽ઃ + υΩϡϝϯτੜ੒ػೳΛݕ౼ 33 ࣮ࡍͷϢʔεέʔε ૝ఆ֎ͷϢʔεέʔε1: υΩϡϝϯτͱͯ͠
34. • ؍ଌͰ͖ͨϢʔεέʔε͸ѹ౗తʹʮGoͷςετίʔυ΁ͷ૊ΈࠐΈʯͰ͸ͳ͘ ʮrunnίϚϯυʢπʔϧʣͱͯ͠ͷར༻ʯͩͬͨ • ςετίʔυ΁ͷ૊ΈࠐΈΑΓ΋ʮμ΢ϯϩʔυͨ͠Β͙͢࢖͑Δʯͷ΄͏͕ ѹ౗తʹָ • GoʹݶΒͳ͍൚༻తͳར༻͕Մೳ • ػೳͷఏҊ΍۩ମతͳػೳ௥ՃͷPull Request΋΋Βͬͨ + runnίϚϯυଆͷػೳ΋ϑΝʔετϓϥΠΦϦςΟͱ֦ͯ͠ॆ͍ͤͯͬͨ͞ 34 ࣮ࡍͷϢʔεέʔε ૝ఆ֎ͷϢʔεέʔε2: runnίϚϯυͷ׆༻
35. • ΞϓϦέʔγϣϯσϓϩΠ௚ޙͷಈ࡞֬ೝͷͨΊͷ؆қతͳϦάϨογϣϯςετͷ ඞཁੑ • ϖύϘ͸͞·͟·ͳٕज़ελοΫͰߏங͞Εͨଟ͘ͷαʔϏεΛӡ༻͍ͯ͠Δ • ؆қͰྑ͍ɻ͔͠͠ɺʮϒϥ΢βϕʔεʯʮ։ൃݴޠʹدΒͳ͍͜ͱʯʮηοτ Ξοϓ͕؆୯Ͱ͋Δ͜ͱʯʮςετ࡞੒ֶ͕शίετ͕খ͍͜͞ͱʯ͕ཁ݅ + Chrome DevTools ProtocolʹରԠ • ChromeʢChromiumʣࠐΈͷDocker imageΛ༻ҙ͢Δ͜ͱͰ্هཁ݅Λຬͨͨ͠ • docker pull ghcr.io/k1low/runn:latest 35 ࣮ࡍͷϢʔεέʔε ૝ఆ֎ͷϢʔεέʔε3: ϦάϨογϣϯςετ
36. • ૝ఆ֎ͷϢʔεέʔεΛ౿·͑ͯɺ͋ΒͨΊͯ • runn ( means "Run N". is pronounced /rʌ́n én/. ) is a package/tool for running operations following a scenario. • ʮΦϖϨʔγϣϯࣗಈԽʯΛπʔϧͷλʔήοτͱ͢ΔͷͰ͋Ε͹ɺʢ౰ॳ͔Β͢ Δͱʣ૝ఆ֎ͳϢʔεέʔε΋෇ਵͨ͠৽ػೳ΋ʮਖ਼౰ͳྲྀΕʯͱ΋͍͑Δ + γφϦΦ࣮ߦϩάͷه࿥ػೳ … ࣮ߦՄೳͳૢ࡞खॱॻͱͯ͠ + SSH Runner … αʔόΦϖϨʔγϣϯࣗಈԽͷखஈͱͯ͠ • ʮ1όΠφϦͰಋೖ͕༰қʯʮՄಡੑߴ͍γφϦΦϑΝΠϧʯͷ2ͭ͸ݫक 36 ࣮ࡍͷϢʔεέʔε ʮΦϖϨʔγϣϯࣗಈԽʯͱ͍͏λʔήοτ
37. ͓ΘΓʹ 37
38. • APIγφϦΦςεςΟϯάπʔϧ/ΦϖϨʔγϣϯࣗಈԽπʔϧͱͯ͠runnΛ঺հ • πʔϧ։ൃऀͱͯ͠૝ఆ͍ͯͨ͠Ϣʔεέʔεͱ࣮ࡍͷϢʔεέʔε͸ɺͣΕΔ͜ͱ ͕͋Δʢ͓ͦΒ͘ྑ͍৔߹΋ѱ͍৔߹΋͋Δʣ • ։ൃऀͱͯ͠͸ɺ͢΂ͯϑΟʔυόοΫͱͯ͠ड͚ࢭΊ͍ͨͱࢥ͍ͬͯΔ • ੈͷπʔϧ։ൃऀ͸େମͦ͏ࢥ͍ͬͯΔʢͨͿΜʣ • ʢ࣌ʑݫ͍͜͠ͱ΋͋Δʢͦͷͱ͖͸͝ΊΜͳ͍͞ʣʣ • ϑΟʔυόοΫ͸خ͍͠ʂ 38 ͓ΘΓʹ ·ͱΊ
39. 39 ͓ΘΓʹ ࠓޙ • ࠓrunnʹ଍Γͳ͍΋ͷͱͯ͠͸ʮ೔ຊޠͷ৘ใʯͩͱߟ͍͑ͯΔ • runnͷػೳʹରͯ͠গͳ͍ • ࡞ऀ͕ॻ͍͍ͯͳ͍ͷͰɺͦΕ͸ͦ͏ • ݱࡏʮrunn cookbookʯΛZennͷbookͱͯ͠४උத • ϨγϐΛඞཁʹԠͯ͡ਵ࣌௥Ճ͍ͯ͘͠ํࣜ • ΋͠ڵຯ͕Ͱ·ͨ͠Βੋඇࢼ͠ʹ࢖ͬͯΈ͍ͯͩ͘͞ʂ • macOSͳΒ brew install k1LoW/tap/runn • LinuxͳΒ deb΍RPM΍apkύοέʔδ • docker pull ghcr.io/k1low/runn:latest
40. Thank You! Thank You! 40