Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Hillel Wayne - Beyond Unit Tests: Taking Your T...

Hillel Wayne - Beyond Unit Tests: Taking Your Testing to the Next Level

You've used pytest and you've used mypy, but bugs are still slipping through your code. What's next? In this talk, we cover two simple but powerful tools for keeping your code problem-free. Property-based testing, provided by the [Hypothesis](https://hypothesis.readthedocs.io/en/latest/) library, lets you run hundreds of tests from a single template. Contracts, via [dpcontracts](https://github.com/deadpixi/contracts), make your program test itself. You'll learn how and why to use these tools and how to combine them with the rest of your testing suite.

https://us.pycon.org/2018/schedule/presentation/130/

PyCon 2018

May 11, 2018
Tweet

More Decks by PyCon 2018

Other Decks in Programming

Transcript

  1. Ingredients • Oats • Honey • Sugar • Butter •

    Chocolate • Coconut • Apricots • Pumpkin Seeds • Oranges • Raisins • Cranberries @hillelogram hillelwayne.com
  2. Just tweet / email / hunt me down during the

    conf hillelwayne.com @hillelogram
  3. def test_works1(): assert add(1, 2) == 3 def test_works2(): assert

    add(0, 0) == 0 def test_works3(): assert add(1, 0) == 1 def test_works4(): assert add(5, -2) == 3 @hillelogram hillelwayne.com
  4. @hillelogram This Photo by Unknown Author is licensed under CC

    BY-SA Example Tests "Automanual" Tests hillelwayne.com
  5. def slope(p1: point, p2: point) @hillelogram This Photo by Unknown

    Author is licensed under CC BY-SA hillelwayne.com
  6. @hillelogram a = 0, b = 1 @given(integers(), integers()) def

    test_commutative(a, b): > assert add(a, b) == add(b, a) E assert 0 == 1 E + where 0 = add(0, 1) E + and 1 = add(1, 0) Falsifying example: test_commutative(a=0, b=1) hillelwayne.com
  7. def add(a: int, b: int) -> int: """Adds two numbers."""

    if '2' in str(b): return 3 return a @hillelogram hillelwayne.com
  8. For a given list, return the maximum possible product of

    two elements in it. max_product([1, 5, 0, 2]) == 10 @hillelogram hillelwayne.com
  9. def max_product(l: List[int]) -> int: if len(l) < 2: raise

    TabError b1, b2 = sorted(l, reverse=True)[0:2] return b1*b2 @hillelogram hillelwayne.com
  10. def test1(): assert max_product([1, 2, 5]) == 10 def test2():

    assert max_product([5, 1, 2, 3]) == 15 def test3(): assert max_product([5, 1, 2, 6, 3, 1]) == 30 def test4(): with raises(TabError): max_product([1]) @hillelogram hillelwayne.com
  11. [ 1 , 5 , 2 , 8 , 3

    , 0 , 1 ] hillelwayne.com @hillelogram
  12. @hillelogram li = [-1, -1, 0] @given(lists(integers(), min_size=2)) def test_always_better(li):

    > assert max_product(li) >= li[0]*li[1] E assert 0 >= (-1 * -1) E + where 0 = max_product([-1, -1, 0]) Falsifying example: test_always_better(li=[-1, -1, 0]) hillelwayne.com
  13. def add(x: int, y: int) -> int: return str(x +

    y) "error: Incompatible return value type (got "str", expected "int")" @hillelogram hillelwayne.com
  14. def add(x: int, y: int) -> int: return str(x +

    y) "error: Incompatible return value type (got "str", expected "int")" @hillelogram hillelwayne.com
  15. def add(x: int, y: int) -> int: return x +

    y def g(): add(1, "2") @hillelogram hillelwayne.com
  16. def add(x, y): #requires assert isinstance(x, int) assert isinstance(y, int)

    z = x + y #ensures assert isinstance(z, int) return z @hillelogram hillelwayne.com
  17. For a given list, return all-but-the first element in it.

    tail([9, 2, 1, "a"]) == [2, 1, "a"] @hillelogram hillelwayne.com
  18. tail([9, 2, 1, "a"]) == [2, 1, "a"] [9] +

    [2, 1, "a"] == [9, 2, 1, "a"] @hillelogram hillelwayne.com
  19. def tail(l: List[Any]) -> List[Any]: output = [] assert [l[0]]

    + output == l return output @hillelogram hillelwayne.com
  20. @hillelogram >>> tail([1,2,3]) Traceback (most recent call last): File "<stdin>",

    line 1, in <module> File "<stdin>", line 3, in tail AssertionError: Postcondition >>> hillelwayne.com
  21. def tail(l: List[Any]) -> List[Any]: output = l[1:] assert [l[0]]

    + output == l return output @hillelogram hillelwayne.com
  22. @hillelogram >>> tail([1,2,3]) [2, 3] >>> tail([]) Traceback (most recent

    call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in tail IndexError: list index out of range >>> hillelwayne.com
  23. def tail(l: List[Any]) -> List[Any]: assert len(l) > 0 output

    = l[1:] assert [l[0]] + output == l return output @hillelogram hillelwayne.com
  24. @hillelogram >>> tail([]) Traceback (most recent call last): File "<stdin>",

    line 1, in <module> File "<stdin>", line 2, in tail AssertionError: Precondition >>> hillelwayne.com
  25. @require("l must not be empty", lambda args: len(args.l) > 0)

    def tail(l: List[Any]) -> List[Any]: assert len(l) > 0 output = l[1:] assert [l[0]] + output == l return output @hillelogram hillelwayne.com
  26. @hillelogram @require("l must not be empty", lambda args: len(args.l) >

    0) @ensure("result is tail of list", lambda args, result: [args.l[0]] + result == args.l) def tail(l: List[Any]) -> List[Any]: output = l[1:] assert [l[0]] + output == l return output hillelwayne.com
  27. @hillelogram @require("l must not be empty", lambda a: len(a.l) >

    0) @ensure("result is tail of list", lambda a, r: [a.l[0]] + r == a.l) def tail(l: List[Any]) -> List[Any]: return l[1:] hillelwayne.com
  28. def is_max_prod(a, r): pairs = combinations(a.l, 2) return all((r >=

    x * y for x, y in pairs)) @hillelogram hillelwayne.com
  29. @require("2+ elements", lambda a: len(a.l) > 1) @ensure("is the max

    prod", is_max_prod) def max_product(l: List[int]) -> int: # efficient version @hillelogram hillelwayne.com
  30. Class Invariants @invariant("No overdrafts", lambda self: self.account >= 0) class

    BankAccount: def withdraw(self, val): ... def deposit(self, val): ... @hillelogram hillelwayne.com
  31. Stateful Testing class NumberModifier(RuleBasedStateMachine): num = 0 @rule(x=integers()) def add_num(x):

    self.num += x @precondition(lambda self: self.num != 0) @rule() def reciprocal(self): self.num = 1 / self.num @hillelogram hillelwayne.com
  32. Crazier Stuff • Test Generators (Clojure) • Contract Inheritance (Eiffel)

    • Global Contracts (Ada) • Static verification (Dafny) • ABC metatyping (Python???) @hillelogram hillelwayne.com
  33. Conclusion @hillelogram Unit Tests => Property Tests Mypy => Contracts

    Integration Tests => Property Tests + Contracts hillelwayne.com
  34. Come to the formal methods open space! Saturday 2 PM

    Room 9 hillelwayne.com @hillelogram