Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Property-Based Testing @ COSCUP...

cybai
July 29, 2022

Introduction to Property-Based Testing @ COSCUP 2022

cybai

July 29, 2022
Tweet

More Decks by cybai

Other Decks in Programming

Transcript

  1. • @CYBAI • @_cybai • Front-End Engineer @ Autify, Inc.

    • Web Front-End by day, learning Functional Programming by night. whoami 3
  2. Agenda 1. What is Property-Based Testing? 2. More examples for

    Property-Based Testing 3. When to Example-Based Testing? 4. Advanced topics? 5. Q & A 4 Note: PBT refers to Property-Based Testing in this talk.
  3. What is Property? 9 A property is an invariant of

    your testing target. What does that mean? Let’s start from the traditional example-based testing.
  4. What is Property-Based Testing? (Cont.) 10 Haskell: QuickCheck, Hedgehog Rust:

    quickcheck, proptest Erlang: PropEr JavaScript: fast-check, jsverify Python: Hypothesis OCaml: popper … and many more PBT frameworks for different languages
  5. 11

  6. What is Property-Based Testing? (Cont.) A test case should be

    like a proof for the invariant for the testing target. 16
  7. A test case should be like a proof for the

    invariant for the testing target. In the example of `reverse` function, the invariant here is “An input array should be kept as same if it’s reversed twice”. What is Property-Based Testing? (Cont.) 17
  8. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

    Shrinkers fc.integer() fc.webUrl() fc.oneof() fc.record() fc.boolean() fc.nat() fc.string() .filter() fc.option() .map() 19 Generate random values
  9. What values will fast-check Arbitraries generate? 20 fc.integer() => -1000,

    -10, …, 0, 1, 3, …, 100, … fc.nat() => 0, 1, 2, …, 100000, …
  10. What values will fast-check Arbitraries generate? 21 fc.integer() => -1000,

    -10, …, 0, 1, 3, …, 100, … fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", …
  11. What values will fast-check Arbitraries generate? 22 fc.integer() => -1000,

    -10, …, 0, 1, 3, …, 100, … fc.oneof(fc.nat(), fc.char()) => Use one of the given arbitraries fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", … .filter() => precondition for the arbitrary (e.g. n < 100) fc.array(fc.nat()) => [], [1], [0,2,2,3], …, [3,5, …, 10,100]
  12. What values will fast-check Arbitraries generate? 23 fc.integer() => -1000,

    -10, …, 0, 1, 3, …, 100, … fc.webUrl() => "https://lo.ui", "https://4.xlm", … fc.oneof(fc.nat(), fc.char()) => Use one of the given arbitraries fc.record({ age: fc.nat(), name: fc.string() }) => Arbitrary for given object model, e.g. { age: 2, name: "John" } fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", … .filter() => precondition for the arbitrary (e.g. n < 100) fc.array(fc.nat()) => [], [1], [0,2,2,3], …, [3,5, …, 10,100] fc.option(fc.nat()) => null, 0, 1, …, 100, …, 100000, …
  13. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

    Shrinkers Run tests for properties and validate specified predicates. Most of PBT frameworks default to 100 number of runs but you can change it. 24
  14. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

    Shrinkers Teach the PBT framework how to shrink the failed test case. This would be helpful for knowing smallest possible case. (but not every PBT framework supports this) 25
  15. More examples for Property-Based Testing You can find more examples

    on Advent of PBT 2021 created by the author of fast-check. 35
  16. When to Example-Based Testing? You can still write unit tests

    in example-based testing way to make people easier to understand how to use them. I’d highly recommend test them with `doctest`. 36
  17. When to Example-Based Testing? Let’s see how `doctest` is utilized

    in Rust. By default, code block in doc comments can be tested by `cargo test` or `rustdoc --test`. So, it can be tested as a part of document. 37 The calculate shipping fee example in Rust
  18. 1. PBT for End-to-End Testing (like this post) 2. Model-based

    testing (like XState testing) Advanced Topics? 40
  19. 1. PBT for End-to-End Testing (like this post) 2. Model-based

    testing (like XState testing) 3. PBT for Web Application (like Quickstrom) Advanced Topics? 41
  20. Advanced Topics? 1. PBT for End-to-End Testing (like this post)

    2. Model-based testing (like XState testing) 3. PBT for Web Application (like Quickstrom) 4. How to specify it! 42
  21. Advanced Topics? 1. PBT for End-to-End Testing 2. Model-based testing

    3. PBT for Web Application (like Quickstrom) 4. How to specify it! 43
  22. Advanced Topics? 1. PBT for End-to-End Testing (like this post)

    2. Model-based testing (like XState testing) 3. PBT for Web Application (like Quickstrom) 4. How to specify it! 5. ... and maybe some more I don’t know yet! 44
  23. Test properties (invariants) with property-based testing and move example-based testing

    as document with `doctest`. Let’s try to prove our programs by Property-Based Testing! Conclusion 46