Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Controllable Randomness in Unit Tests

Controllable Randomness in Unit Tests

Avatar for Oleksandr Voronov

Oleksandr Voronov

April 05, 2019
Tweet

More Decks by Oleksandr Voronov

Other Decks in Programming

Transcript

  1. { "id": 6253282, "id_str": "6253282", "name": "Twitter API", "screen_name": "twitterapi",

    "location": "San Francisco, CA", "url": "https://dev.twitter.com", "description": "The Real Twitter API.", "derived": { "locations": [ {} ] }, "protected": true, "verified": false, "followers_count": 21, "friends_count": 32, "listed_count": 9274, "favourites_count": 13, "statuses_count": 42, "created_at": "Mon Nov 29 21:18:15 +0000 2010", "geo_enabled": true, "lang": "zh-cn", "contributors_enabled": false, "profile_background_color": "e8f2f7", "profile_background_image_url": "http://a2.twimg.com/profile_background_images/229557229/twitterapi-bg.png", "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/229557229/twitterapi-bg.png", "profile_background_tile": false, "profile_banner_url": "https://si0.twimg.com/profile_banners/819797/1348102824", "profile_image_url": "http://abs.twimg.com/sticky/default_profile_images/default_profile_normal.png", "profile_image_url_https": "https://abs.twimg.com/sticky/default_profile_images/default_profile_normal.png", "profile_link_color": "0094C2", "profile_sidebar_border_color": "0094C2", "profile_sidebar_fill_color": "a9d9f1", "profile_text_color": "437792", "profile_use_background_image": true, "default_profile": false, "default_profile_image": false, "withheld_in_countries": [ "GR", "HK", "MY" ], "withheld_scope": "user" } @aleks_voronov • CocoaFriday#2
  2. DATA GENERATED ON THE FLY ! ▸ Specification OpenAPI •

    Swift Type System ▸ Behavior Properties SwiftCheck @aleks_voronov • CocoaFriday#2
  3. MIXED APPROACH Generated by specification • using predefined data sets

    • and manual clarifications @aleks_voronov • CocoaFriday#2
  4. func random(from: Int, to: Int) -> Int { return Int(arc4random_uniform(UInt32(to)

    - UInt32(from) + 1) + UInt32(from)) } @aleks_voronov • CocoaFriday#2
  5. 4.2 protocol RandomNumberGenerator { mutating func next() -> UInt64 ...

    } github.com/apple/swift-evolution/blob/master/proposals/0202-random-unification.md @aleks_voronov • CocoaFriday#2
  6. 4.2 Bool • Int • Double • Array • Set

    • Dictionary random() -> T random<G: RandomNumberGenerator>(inout G) -> T github.com/apple/swift-evolution/blob/master/proposals/0202-random-unification.md @aleks_voronov • CocoaFriday#2
  7. GENERIC 'RANDOM' INTERFACE protocol Random { static func random<G: RandomNumberGenerator>(

    using generator: inout G ) -> Self } @aleks_voronov • CocoaFriday#2
  8. CONFORMING BASICS extension Character: Random { ... } extension UnicodeScalar:

    Random { ... } extension FloatingPoint where Self: Random { ... } extension FixedWidthInteger where Self: Random { ... } @aleks_voronov • CocoaFriday#2
  9. extension Bool: Random { } extension Float: Random { }

    extension Double: Random { } extension Int: Random { } extension Int64: Random { } extension Int32: Random { } extension Int16: Random { } extension Int8: Random { } extension UInt: Random { } extension UInt64: Random { } extension UInt32: Random { } extension UInt16: Random { } extension UInt8: Random { } @aleks_voronov • CocoaFriday#2
  10. RANDOM COLLECTIONS extension Array: Random where Element: Random { static

    func random<G: RandomNumberGenerator>( using generator: inout G ) -> [Element] { ... } } @aleks_voronov • CocoaFriday#2
  11. RANDOM ENUMS protocol RandomAll: Random { static func allRandom<G: RandomNumberGenerator>(

    using generator: inout G ) -> [Self] } @aleks_voronov • CocoaFriday#2
  12. OPTIONAL EXAMPLE extension Optional: RandomAll where Wrapped: Random { public

    static func allRandom<G: RandomNumberGenerator>( using generator: inout G ) -> [Wrapped?] { return [.none, .some(.random(using: &generator))] } } extension Optional: Random where Wrapped: Random {} @aleks_voronov • CocoaFriday#2
  13. RANDOM TUPLES func randomTuple< A: Random, B: Random, G: RandomNumberGenerator

    >(using generator: inout G) -> (A, B) { return ( .random(using: &generator), .random(using: &generator) ) } @aleks_voronov • CocoaFriday#2
  14. LET'S USE IT struct User { let id: String let

    name: String let age: Int let friends: (followers: Int, followees: Int) } @aleks_voronov • CocoaFriday#2
  15. extension User: Random { static func random<G: RandomNumberGenerator>( using generator:

    inout G ) -> User { return User( id: .random(using: &generator), name: .random(using: &generator), age: .random(using: &generator), friends: randomTuple(using: &generator) ) } } @aleks_voronov • CocoaFriday#2
  16. struct PRNG: RandomNumberGenerator { typealias State = UInt64 private(set) var

    state: State init() { var generator = SystemRandomNumberGenerator() state = .random(using: &generator) } init(seed: State) { state = seed } mutating func next() -> UInt64 { state = magic(state) return state } } @aleks_voronov • CocoaFriday#2
  17. If an NSPrincipalClass key is declared in the test bundle's

    Info.plist file, XCTest automatically creates a single instance of that class when the test bundle is loaded ... – developer.apple.com/documentation/... @aleks_voronov • CocoaFriday#2
  18. public var R = PRNG() class RandomSeed: NSObject { override

    init() { super.init() let seed = "\(R.state)" print(seed) } } @aleks_voronov • CocoaFriday#2
  19. GET EVERY FAILED TEST SEED ✅ ✅ ❌ ✅ ❌

    –––––––8–––––12–––––––––––23–42 @aleks_voronov • CocoaFriday#2
  20. If an NSPrincipalClass key is declared in the test bundle's

    Info.plist file, XCTest automatically creates a single instance of that class when the test bundle is loaded. You can use this instance as a place to register observers or do other pretesting global setup before testing for that bundle begins. – developer.apple.com/documentation/xctest/xctestobservationcenter @aleks_voronov • CocoaFriday#2
  21. XCTESTOBSERVATION ▸ START: Save Seed 1 ▸ FAIL: Log Seed

    * ▸ SUCCESS: Do nothing * developer.apple.com/documentation/xctest/xctestobservation @aleks_voronov • CocoaFriday#2
  22. import Foundation {# So far getting imports hardcoded from config,

    correct approach might be found here: https://github.com/krzysztofzablocki/Sourcery/issues/670 #} {% for import in argument.imports %} import {{ import }} {% endfor %} {% if argument.testable %}{% for testable in argument.testable %} @testable import {{ testable }} {% endfor %}{% endif %} {% macro randomValue type %}{% if type.kind == "protocol" %}{{ type.inheritedTypes.0.name }}{% endif %}{% endmacro %} // MARK: - Structs {# Random Struct #} {% macro rng %}&{{ argument.rng }}{% endmacro %} {% macro customRandomValueType variable %}{% if variable.annotations.random %}{{ variable.annotations.random }}{% endif %}{% endmacro %} {% macro randomValueUsingGenerator variable %}{% call customRandomValueType variable %}{% if variable.isTuple %}randomTuple(using: &generator){% else %}.random(using: &generator){% endif %}{% endmacro %} {% macro randomValueUsingRNG variable %}{% call customRandomValueType variable %}{% if variable.isTuple %}randomTuple(using: {% call rng %}){% else %}.random(using: {% call rng %}){% endif %}{% endmacro %} {% for type in types.structs where type|annotated:"Random" %} extension {{ type.name }}: Random { public static func random<G: RandomNumberGenerator>(using generator: inout G) -> {{ type.name }} { return {{ type.name }}( {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: {% call randomValueUsingGenerator variable %}{% if not forloop.last %},{% endif %} {% endfor %} ) } {% if argument.rng %} {# user-friendly version, so that you don't have to bother with closures and incoming generators. But it's tightly coupled to `rng` argument value from sourcery config #} public static func random( {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: {{ variable.typeName }} = {% call randomValueUsingRNG variable %}{% if not forloop.last %},{% endif %} {% endfor %} ) -> {{ type.name }} { return {{ type.name }}( {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: {{ variable.name }}{% if not forloop.last %},{% endif %} {% endfor %} ) } {% else %} {# this one is generated additionally, so that you can have everything random except for some selected fields #} public static func random<G: RandomNumberGenerator>( _ generator: inout G, {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: (inout G) -> {{ variable.typeName }} = { generator in {% call randomValueUsingGenerator variable %} }{% if not forloop.last %},{% endif %} {% endfor %} ) -> {{ type.name }} { return {{ type.name }}( {% for variable in type.variables where not variable.isComputed %} {{ variable.name }}: {{ variable.name }}(&generator){% if not forloop.last %},{% endif %} {% endfor %} ) } {% endif %} } {% endfor %} // MARK: - Enums {# Random Enum #} {% macro randomEnumCaseUsingGenerator case %}.{{ case.name }}{% if case.hasAssociatedValue %}({% for value in case.associatedValues %}{% if value.localName %}{{ value.localName }}: {% endif %}{% call randomValueUsingGenerator value %}{% if not forloop.last %}, {% endif %}{% endfor %}){% endif %}{% endmacro %} {% macro randomIfCaseUsingGenerator case %}{% if case.hasAssociatedValue %}{{ case.name }}(&generator){% else %}.{{ case.name }}{% endif %}{% endmacro %} {% macro randomEnumCaseUsingRNG case %}.{{ case.name }}{% if case.hasAssociatedValue %}({% for value in case.associatedValues %}{% if value.localName %}{{ value.localName }}: {% endif %}{% call randomValueUsingRNG value %}{% if not forloop.last %}, {% endif %}{% endfor %}){% endif %}{% endmacro %} {% macro randomIfCaseUsingRNG case %}{% if case.hasAssociatedValue %}{{ case.name }}{% else %}.{{ case.name }}{% endif %}{% endmacro %} {% for enum in types.enums where enum|annotated:"Random" %} {% if enum.cases.count == 0 %} #warning("`{{ enum.name }}` is an uninhabitant type and no value of it can be created, thus it can't have random value as well.") // extension {{ enum.name }}: Random { } {% elif not enum.hasAssociatedValues %} extension {{ enum.name }}: Random { {# I'd expect to check `enum.based` property, but it's always empty ¯\_(ϑ)_/¯ #} {% if enum.rawTypeName.name != "CaseIterable" and enum.inheritedTypes|join:" "|!contains:"CaseIterable" %} #warning("Please, conform to `CaseIterable` protocol, so that compiler takes care of this") public static let allCases: [{{ enum.name }}] = [{% for case in enum.cases %}.{{ case.name }}{% if not forloop.last %}, {% endif %}{% endfor %}] {% endif %} public static func random<G: RandomNumberGenerator>(using generator: inout G) -> {{ enum.name }} { return allCases.randomElement(using: &generator)! } } {% else %} extension {{ enum.name }}: RandomAll { public static func allRandom<G: RandomNumberGenerator>(using generator: inout G) -> [{{ enum.name }}] { {% if enum.cases.count == 1 %} return [{% call randomEnumCaseUsingGenerator enum.cases.0 %}] {% else %} return [{% for case in enum.cases %} {% call randomEnumCaseUsingGenerator case %}{% if not forloop.last %},{% endif %}{% endfor %} ] {% endif %} } {# alternative version, so that you don't have to bother with closures and incoming generators but it's tightly coupled to `rng` argument value from sourcery config and it executes and calculates randoms for all cases even though only one will be needed (thus a bit more expensive) #} {% if argument.rng %} public static func random( {% for case in enum.cases where case.associatedValues %} {{ case.name }}: {{ enum.name }} = {% call randomEnumCaseUsingRNG case %}{% if not forloop.last %},{% endif %} {% endfor %} ) -> {{ enum.name }} { {% if enum.cases.count == 1 %} return {% call randomIfCaseUsingRNG enum.cases.0 %} {% else %} return [ {% for case in enum.cases %} {% call randomIfCaseUsingRNG case %}{% if not forloop.last %},{% endif %} {% endfor %} ].randomElement(using: {% call rng %})! {% endif %} } {% else %} {# same for enum - you can override `random` for some cases and leave others generated by default #} public static func random<G: RandomNumberGenerator>( _ generator: inout G, {% for case in enum.cases where case.associatedValues %} {{ case.name }}: (inout G) -> {{ enum.name }} = { generator in {% call randomEnumCaseUsingGenerator case %} }{% if not forloop.last %},{% endif %} {% endfor %} ) -> {{ enum.name }} { {% if enum.cases.count == 1 %} return {% call randomIfCaseUsingGenerator enum.cases.0 %} {% else %} return [ {% for case in enum.cases %} {% call randomIfCaseUsingGenerator case %}{% if not forloop.last %},{% endif %} {% endfor %} ].randomElement(using: &generator)! {% endif %} } {% endif %} } {% endif %} {% endfor %} @aleks_voronov • CocoaFriday#2
  23. INTERESTING ▸ pointfree.co : ep30 - ep32 • ep47 -

    ep50 ▸ github.com/pointfreeco/swift-gen ▸ github.com/typelift/SwiftCheck ▸ ieeexplore.ieee.org/document/6963470 : Oracles @aleks_voronov • CocoaFriday#2