Share
Embed
Property-based testing with SwiftCheck/PBT-with-SwiftCheck
Property-based testing with SwiftCheck
Yusuke Hosonuma
July 27, 2017
Transcript
72 slides
-
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. iOS Test Night
#5 2017/07/26(Wed) ࡉপ༞հ@DeNA 1SPQFSUZCBTFEUFTUJOH XJUI4XJGU$IFDL 1 -
˙:VTVLF)PTPOVNBʢ!UPCJʣ • ॴଐ ⁃ %F/"48&5άϧʔϓςετج൫νʔϜ • ܦྺ ⁃ J04ΤϯδχΞ⾣݄ʹ%F/"48&5άϧʔϓʹ+PJO •
ීஈͷۀ ⁃ όʔδϣϯɾΞοϓࣗಈݕূγεςϜʢϚεςΟϑʣͷ։ൃ ⁃ ΞϓϦͷςετʹ·ͭΘΔௐ͍ࠪΖ͍Ζ • ଞ ⁃ ͱ͔͖Ͱ͢ ࣗݾհ Copyright (C) DeNA Co.,Ltd. All Rights Reserved. 2 -
ΞδΣϯμ 5 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
1SPQFSUZCBTFEUFTUJOHͱʁ • 4XJGU$IFDLͱʁ • ۩ମྫͰݟΔ • ྫ ͠ࢉ • ྫ ྻͷస • ༻ޠ • ࣮ફతͳϢʔεέʔε • ·ͱΊ -
Property-based testing ͱʁ 7 Copyright (C) DeNA Co.,Ltd. All Rights
Reserved. • ଟͳϥϯμϜΛ'8ʹࣗಈੜͤ͞ɺ • ςετରͷੑ࣭ʢ1SPQFSUZʣΛίʔυͰఆٛ͠ɺ • ͦͷੑ࣭͕ഁΒΕΔ͕ͳ͍͔ݕূ͢Δ • Ϣχοτςετख๏ͷͭ • ؔܕݴޠ)BTLFMMͷʮ2VJDL$IFDLʯ͕ݩ -
4XJGU$IFDLͱʁ 9 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
2VJDL$IFDLͷ4XJGU࣮ • IUUQTHJUIVCDPNUZQFMJGU4XJGU$IFDL • 9$5FTUʹΈࠐΊΔ • 4XJGU[Λ࡞͍ͬͯΔ5ZQF-JGUͷϦϙδτϦ • 4XJGU[ͷςετʹΘΕͯΔ • .*5MJDFOTF -
10 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • 1SPQFSUZCBTFEUFTUJOHͱʁ
• 4XJGU$IFDLͱʁ • ۩ମྫͰݟΔ • ྫ ͠ࢉ • ྫ ྻͷస • ༻ޠ • ࣮ફతͳϢʔεέʔε • ·ͱΊ -
// ϓϩμΫτίʔυ func add(_ x: UInt, _ y: UInt) ->
UInt { return x + y } // ςετίʔυ func test_add_by_assert() { XCTAssertEqual(add(1, 2), 3) } XCTest Ͱͷ͠ࢉͷςετ 13 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. 1 + 2 ͷ݁Ռ 3 Ͱ͋Δ͜ͱ • ҰൠతͳY6OJU͕ఏڙ͍ͯ͠Δํ๏ • 2VJDLͳͲͷ#%%ͳ'8ຊ࣭తʹಉ͡ -
ਤઆ 14 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ̍ʴ̎ʹ̏
ೖྗ ೖྗ ظ ςετର -
ਤઆ 15 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ̍ʴ̎ʹ̏
ೖྗ ೖྗ ظ ςετର ೖྗͱظɺ ։ൃऀ͕໌ࣔతʹબͿඞཁ͕͋Δ -
ςετ༻ͷೖྗΛબͿٕज़ 17 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
ಉׂ๏ • ڥքੳ • ΫϥγϑΟέʔγϣϯπϦʔ • 1BJSXJTF๏ • FUDʜ -
ςετ༻ͷೖྗΛબͿٕज़ 18 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
ಉׂ๏ • ڥքੳ • ΫϥγϑΟέʔγϣϯπϦʔ • 1BJSXJTF๏ • FUDʜ ͲΕ֬తʹόάͷى͖͍͢σʔ λΛ༧ଌͯ͠ɺಛఆͷΈͰςετ -
͠ࢉͷੑ࣭ 21 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
݁߹๏ଇΛຬͨ͢ • Y ZZ Y • Λճͨ͠ͷͱɺΛճͨ͠ͷ͍͠ • Y Y • ʢͪͳΈʹʣҾ͖ࢉ݁߹๏ଇΛຬͨ͞ͳ͍ • YZ㱠ZY -
͠ࢉͷੑ࣭ 22 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
݁߹๏ଇΛຬͨ͢ • Y ZZ Y • Λճͨ͠ͷͱɺΛճͨ͠ͷ͍͠ • Y Y • ʢͪͳΈʹʣҾ͖ࢉ݁߹ੑ͋Γ • YZ㱠ZY ͜ΕΒ͕ͲΜͳೖྗʹରͯ͠ อͨΕΔ͖ੑ࣭Ͱ͋Δɺͱߟ͑Δ -
4XJGU$IFDLͰͷ͠ࢉͷੑ࣭ςετ 24 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. //
;ͭ͏ͷXCTestͱಉ͡Α͏ʹ಄ʹ`test`Λ͚ͭΔ func test_add_property() { // x + y == y + x property("͠ࢉ݁߹๏ଇΛຬͨ͢") <- forAll { (x: UInt, y: UInt) in return add(x, y) == add(y, x) } // x + 1 + 1 == x + 2 property("1Λ2ճͨ͠ͷͱɺ2Λ1ճͨ͠ͷ͍͠") <- forAll { (x: UInt) in return add(add(x, 1), 1) == add(x, 2) } } -
4XJGU$IFDLͰͷ͠ࢉͷੑ࣭ςετ 25 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. func
test_add_property() { // x + y == y + x property("͠ࢉ݁߹๏ଇΛຬͨ͢") <- forAll { (x: UInt, y: UInt) in return add(x, y) == add(y, x) } } ᶃੑ࣭ͷએݴ -
4XJGU$IFDLͰͷ͠ࢉͷੑ࣭ςετ 26 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. func
test_add_property() { // x + y == y + x property("͠ࢉ݁߹๏ଇΛຬͨ͢") <- forAll { (x: UInt, y: UInt) in return add(x, y) == add(y, x) } } ᶄҙͷೖྗ -
4XJGU$IFDLͰͷ͠ࢉͷੑ࣭ςετ 27 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. func
test_add_property() { // x + y == y + x property("͠ࢉ݁߹๏ଇΛຬͨ͢") <- forAll { (x: UInt, y: UInt) in return add(x, y) == add(y, x) } } ᶅશͯʹର͠ -
4XJGU$IFDLͰͷ͠ࢉͷੑ࣭ςετ 28 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. func
test_add_property() { // x + y == y + x property("͠ࢉ݁߹๏ଇΛຬͨ͢") <- forAll { (x: UInt, y: UInt) in return add(x, y) == add(y, x) } } ᶆ͜ͷ͕ࣜΓཱͭ -
4XJGU$IFDLͰͷ͠ࢉͷੑ࣭ςετ 29 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. func
test_add_property() { // x + y == y + x property("͠ࢉ݁߹๏ଇΛຬͨ͢") <- forAll { (x: UInt, y: UInt) in return add(x, y) == add(y, x) } } ᶃੑ࣭ͷએݴ ᶄҙͷೖྗ ᶅશͯʹର͠ ᶆ͜ͷ͕ࣜΓཱͭ -
Test Case '-[SwiftCheckSampleTests.SwiftCheckSampleTests test_add_property]' started. *** Passed 100 tests .
*** Passed 100 tests . Test Case '-[SwiftCheckSampleTests.SwiftCheckSampleTests test_add_property]' passed (0.009 seconds). 4XJGU$IFDLͰͷςετ͕ޭ 31 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ίϯιʔϧ 1BTTFEUFTUT ݸͷϥϯμϜͰςετ͕͞Ε͍ͯΔʂ -
۩ମతʹબΕ͍ͯͨ 33 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Test
Case '-[SwiftCheckSampleTests.SwiftCheckSampleTests test_add_property]' started. x: 0, y: 0 x: 1, y: 1 x: 0, y: 2 x: 3, y: 3 x: 0, y: 0 x: 1, y: 1 x: 0, y: 6 x: 4, y: 1 … x: 26, y: 19 x: 34, y: 51 x: 40, y: 18 x: 10, y: 13 x: 49, y: 22 x: 9, y: 14 x: 31, y: 65 x: 45, y: 15 x: 30, y: 32 x: 46, y: 54 x: 65, y: 34 x: 57, y: 41 … x: 58, y: 17 x: 67, y: 17 x: 41, y: 93 x: 21, y: 17 *** Passed 100 tests ϥϯμϜ͚ͩΕͲɺ খ͞Ίͷ͔Βςετ͞Ε͍ͯΔ -
*** Failed! Proposition: x + 1 + 1 == x
+ 2 Falsifiable (after 3 tests and 2 shrinks): 1 *** Passed 2 tests 4XJGU$IFDLͰͷςετ͕ޭ 35 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ίϯιʔϧ ͕ʮʯͷέʔεͰɺ ςετ͕ύε͠ͳ͔ͬͨ͜ͱ͕ใࠂ͞Εͨ -
9$5"TTFSU&RVBMͰݕূ 37 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ͔֬ʹςετ͕ࣦഊͨ͠ʂ
func add(_ x: UInt, _ y: UInt) -> UInt { return x * y } ݪҼֻ͚ࢉʹͳ͍ͬͯͨҝ -
38 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • 1SPQFSUZCBTFEUFTUJOHͱʁ
• 4XJGU$IFDLͱʁ • ۩ମྫͰݟΔ • ྫ ͠ࢉ • ྫ ྻͷస • ༻ޠ • ࣮ફతͳϢʔεέʔε • ·ͱΊ -
ྻͷసͷੑ࣭ͱʁ 41 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ˣస
SFWFSTF ˣస SFWFSTF ճసͤ͞ΔͱݩʹΔʂ -
func test_reversed() { property("ྻͷసͷసݩͷྻͱಉ͡") <- forAll { (xs: ArrayOf<Int>) in
let ys = xs.getArray return ys.reversed().reversed() == ys } } 4XJGU$IFDLͰྻͷసͷςετ 43 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. YT<5> ͱදݱͰ͖ͳ͍ͷ4XJGUͷ੍ݶɻ ΘΓʹ"SSBZ0G5ͱ͍͏ϥούʔΛࢦఆ͠ɺ HFU"SSBZͰதΛऔΓग़͠ɻ "SSBZ0G*OU -
͜͜·Ͱͷ·ͱΊ 45 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
1SPQFSUZCBTFEUFTUJOH • ςετͱͯ͠ੑ࣭ʢQSPQFSUZʣΛఆٛ • '8͕܁Γฦ͠ϥϯμϜͰςετΛ࣮ߦ • 4XJGU$IFDL • 9$5FTUͱΈ߹Θͤͯ͑Δ • ࣦഊͨ͠߹ɺͦͷΛใࠂͯ͘͠ΕΔ • ͦͷͰ͖Δ͚ͩখ͍͕͞બΕ͍ͯΔ -
46 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • 1SPQFSUZCBTFEUFTUJOHͱʁ
• 4XJGU$IFDLͱʁ • ۩ମྫͰݟΔ • ྫ ͠ࢉ • ྫ ྻͷస • ༻ޠ • ࣮ફతͳϢʔεέʔε • ·ͱΊ -
Arbitrary 49 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
ʮҙͷʯͱ͍ͬͨҙຯ • 4XJGU$IFDLͰϓϩτίϧͱͯ͠ఆٛ • ಛఆͷܕͷϥϯμϜΛੜ͢Δׂ • 4XJGUඪ४ͷܕʹσϑΥϧτ࣮͕ఏڙ͞ΕΔ • ͜Εʹ४ڌ͢Δ͜ͱͰϥϯμϜ͕ੜͰ͖ΔΑ͏ʹ public protocol Arbitrary { public static var arbitrary: SwiftCheck.Gen<Self> { get } public static func shrink(_: Self) -> [Self] } -
Shrinking 51 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
ʮऩॖ͢Δʯͱ͍ͬͨҙຯ • ಉ͘͡"SCJUSBSZϓϩτίϧʹఆٛʢPQUJPOBMʣ • ΑΓখ͞ͳʹ͢Δׂ • ͜ΕʹΑΓࣦഊͨ͠ςετσʔλʹ͍ͭͯɺ ΑΓখ͞ͳࣦഊέʔεΛબग़͢ΔΈ Int.shrink(10) // => [0, 1, 2, 5] String.shrink(“app") // => ["p", "pp", ... ,"apa", "apb"] -
Shrinking 52 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
ʢࣦഊʣ ˣTISJOL • ʢࣦഊʣ ˣTISJOL • ʢࣦഊʣ⾣ɹ͜ΕΛʢ࠷খͷʣࣦഊྫͱͯ͠ใࠂ ˣTISJOL • ʢޭʣ -
53 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • 1SPQFSUZCBTFEUFTUJOHͱʁ
• 4XJGU$IFDLͱʁ • ۩ମྫͰݟΔ • ྫ ͠ࢉ • ྫ ྻͷస • ༻ޠ • ࣮ફతͳϢʔεέʔε • ·ͱΊ -
Property-based testing ͕༗ޮͳέʔε 55 Copyright (C) DeNA Co.,Ltd. All Rights
Reserved. • ᶃ&ODPEF%FDPEF • ᶄ͘໌֬ͳΞϧΰϦζϜWTߴͳΞϧΰϦζϜ • ᶅϥϯμϜੑʹج͍ͮͨΞϧΰϦζϜ -
Encode / Decode ͷϢχοτςετ 57 Copyright (C) DeNA Co.,Ltd. All
Rights Reserved. • ΘΓͱେม • ϓϩύςΟ͕ଟ͍ͱॻ͘ͷେม • ಉ͡ೖྗΛ͏ͱೖΕସ͑Λݕग़Ͱ͖ͳ͍ • ςετΛॻ͘࿑ྗʹݟ߹͍ͬͯΔ͔ʁ • ϦεΫ͕ߴ͍ • Ұݟਖ਼͘͠ಈ͍͍ͯΔΑ͏ʹΈ͑Δ • ͚ΕͲɺҰ෦ͷ߲͚͚͍ͩܽͯͨΓ -
Encode / Decode ͷੑ࣭ 58 Copyright (C) DeNA Co.,Ltd. All
Rights Reserved. • &ODPEF%FDPEFͨ͠ͷݩͱಉ͡ • EFDPEF FODPEF PCK PCK -
Dogߏମ 59 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. public
struct Dog: Equatable { let name: String let age: Int func encode() -> [String: String] { return [ "name": name, "age": "\(age)", ] } static func decode(_ encoded: [String: String]) -> Dog? { guard let name = encoded["name"], let age = (encoded["age"].flatMap{ Int($0) }) else { return nil } return Dog(name: name, age: age) } } public func == (a: Dog, b: Dog) -> Bool { return a.name == b.name && a.age == b.age } -
Dog Λ Arbiratyϓϩτίϧʹ४ڌͤ͞Δ 60 Copyright (C) DeNA Co.,Ltd. All Rights
Reserved. extension Dog: Arbitrary { public static var arbitrary: SwiftCheck.Gen<Dog> { return Gen<(String, Int)> .zip(String.arbitrary, Int.arbitrary) .map(Dog.init) } } ࠓճTISJOLͷׂ࣮Ѫ ʢͳͯ͘ಈ͖͢Δʣ -
SwiftCheckͰͷςετίʔυ 61 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. func
testDogEncodeDecode() { property(“DogΛencodeͯ͠decodeͨ͠ͷɺݩͱಉ͡") <- forAll { (dog: Dog) in guard let decoded = Dog.decode(dog.encode()) else { return false } return dog == decoded } } "SCJUSBSZʹ४ڌͨ͠ͷͰɺ ϥϯμϜ͕ੜͰ͖ΔΑ͏ʹͳͬͨ -
༷ʑͳσʔλͰςετ͞Ε͍ͯΔ 62 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Dog(name:
"", age: 0) Dog(name: "O", age: 1) Dog(name: "d", age: 1) Dog(name: "ÇÆØ", age: 3) Dog(name: "!\\-", age: -4) Dog(name: "0S=ä", age: 3) Dog(name: "¡¨sóø", age: 1) Dog(name: "Ü+_<M", age: 4) Dog(name: "fín«ÄqÙv", age: -5) Dog(name: "ÁZõXü:¼¢", age: -4) Dog(name: "Ͼâñ. øýí", age: -9) Dog(name: "ùÝofNÁ&S", age: 9) Dog(name: " aÞäÈyÚ&Á", age: -5) Dog(name: "_¯¼Ì6<ÙRçv", age: 3) Dog(name: "êð©[ý#ùTó}°Õ|É", age: 7) Dog(name: "KVGH<ø«zE7\'", age: 12) -
͘໌֬ͳΞϧΰϦζϜ vs ߴͳΞϧΰϦζϜ 64 Copyright (C) DeNA Co.,Ltd. All Rights
Reserved. • ΞϧΰϦζϜ • ໌֬ͳॻ͖ํΛ͢Δͱʢ͍͍ͩͨʣ͍ • ߴʹ͢Δͱʢ͍͍ͩͨʣίʔυ͕ෳࡶʹͳΔ • ςετͷ͠͞ • +BWBͷ4%,ʹؚ·ΕΔೋ୳ࡧɺ ؒόάʹؾ͔ͮΕͳ͔ͬͨ -
͘໌֬ͳΞϧΰϦζϜ vs ߴͳΞϧΰϦζϜ 65 Copyright (C) DeNA Co.,Ltd. All Rights
Reserved. • ੑ࣭Λߟ͑Δ • ೖྗ͕ಉ͡Ͱ͋Εɺ྆ऀͷ݁Ռৗʹಉ͡ -
֊ΛٻΊΔίʔυ 66 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. //
֊Λ࠶ؼͰٻΊΔ func fact(_ n: Int) -> Int { if n == 1 { return 1 } return n * fact(n - 1) } // ֊ΛϧʔϓͰٻΊΔ func factFast(_ n: Int) -> Int { var r = n var i = n - 1 while i > 1 { r *= i i = i - 1 } return r } ࠶ؼ൛͍͕ٙͳ͍͘Β͍ʹ໌֬ɻ͚ΕͲίʔϧ ελοΫͷɺύϑΥʔϚϯεѱ͍ -
SwiftCheckͰͷςετίʔυ 67 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. func
test_fact() { // 1ʙ20ͷΛબͿʢେ͖͗͢ΔͱΤϥʔʹͳΔͨΊʣ let gen: Gen<Int> = Gen<Int>.choose((1, 20)) property("fact()ͱfactFast()ৗʹಉ݁͡ՌΛฦ͢͜ͱ") <- forAll(gen) { (n: Int) in return fact(n) == factFast(n) } } ໌֬ͳΞϧΰϦζϜͷ݁ՌͱҰக͍ͯ͠Εɺߴ ͳΞϧΰϦζϜਖ਼͘͠ػೳ͍ͯ͠Δͱߟ͑Β ΕΔ DIPPTFͰͷൣғΛ੍ݶ -
ϥϯμϜੑʹج͍ͮͨΞϧΰϦζϜ 69 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
έʔε • μϯδϣϯϚοϓͷੜ • μΠεϩʔϧ • ੑ࣭Λߟ͑Δ • Ϛοϓඞͣΰʔϧʹͭͳ͕͍ͬͯΔ͜ͱ • Ϛοϓʹ౸ୡͰ͖ͳ͍෦͕ͳ͍͜ͱ • ेʹۉͳϥϯμϜੑ͕ಘΒΕΔ͜ͱ -
·ͱΊ 71 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •
1SPQFSUZCBTFEUFTUJOH • ੑ࣭ʢQSPQFSUZʣʹணͨ͠Ϣχοτςετख๏ • ଟͳϥϯμϜͰςετ͞ΕΔ • 4XJGU$IFDLͰ9$5FTUͱΈ߹Θͤͯ͑Δ • ͍Ͳ͜Ζ • ࣍ୈͰ͍Ζ͍Ζ͋Γͦ͏ • &YBNQMFCBTFEUFTUJOHͱྑ͘