Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Matt Bachmann - Better Testing With Less Code: ...

Matt Bachmann - Better Testing With Less Code: Property Based Testing With Python

Standard unit tests have developers test specific inputs and outputs. This works, but often what breaks code are the cases we did not think about. Property based testing has developers define properties of output and has the computer explore the possible inputs to verify these properties. This talk will introduce property based testing and provide real world examples and patterns.

https://us.pycon.org/2016/schedule/presentation/1927/

PyCon 2016

May 29, 2016
Tweet

More Decks by PyCon 2016

Other Decks in Programming

Transcript

  1. def test_sort_empty(xs): assert quicksort([]) == [] def test_sorted(xs): assert quicksort([1,

    2, 3]) == [1, 2, 3] def test_sort_unsorted(xs): assert quicksort([5, 4]) == [4, 5]
  2. Property Based Testing • Describe the arguments • Describe the

    result • Have the computer try to prove your code wrong
  3. • The Arguments ◦ List of Integers • Properties of

    the Result ◦ List ◦ All elements preserved ◦ Results are in ascending order
  4. @given(st.lists(st.integers())) def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert

    Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in zip(sorted_xs, sorted_xs[1:]) )
  5. @given(st.lists(st.integers())) def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert

    Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in zip(sorted_xs, sorted_xs[1:]) )
  6. @given(st.lists(st.integers())) def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert

    Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in zip(sorted_xs, sorted_xs[1:]) )
  7. @given(st.lists(st.integers())) def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert

    Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in zip(sorted_xs, sorted_xs[1:]) )
  8. @given(st.lists(st.integers())) def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert

    Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in zip(sorted_xs, sorted_xs[1:]) )
  9. @given(st.lists(st.integers())) def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert

    Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in zip(sorted_xs, sorted_xs[1:]) )
  10. @given(st.lists(st.integers())) def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert

    Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in zip(sorted_xs, sorted_xs[1:]) )
  11. @given( st.builds( Dog, breed=st.sampled_from(KNOWN_BREEDS), name=st.text(min_size=5), height=st.floats( min_value=1, max_value=6, allow_nan=False, allow_infinity=False

    ), weight=st.floats( min_value=1, max_value=300, allow_nan=False, allow_infinity=False ), ) ) @example(Dog(breed="Labrador", name="Spot", height=2.1, weight=70))
  12. @given(st.integers(), st.text(), st.integers())) def test_no_explosion(id, sort, max): response = requests.get(

    BATPUTER_URL.format(id, sort, max) ) assert response and response.json() assert (response.status_code in [200, 401, 400, 404])
  13. @given(st.integers(), st.text(), st.integers())) def test_no_explosion(id, sort, max): response = requests.get(

    BATPUTER_URL.format(id, sort, max) ) assert response and response.json() assert (response.status_code in [200, 401, 400, 404])
  14. @given(st.integers(), st.text(), st.integers())) def test_no_explosion(id, sort, max): response = requests.get(

    BATPUTER_URL.format(id, sort, max) ) assert response and response.json() assert (response.status_code in [200, 401, 400, 404])
  15. @given(st.integers(), st.text(), st.integers())) def test_no_explosion(id, sort, max): response = requests.get(

    BATPUTER_URL.format(id, sort, max) ) assert response and response.json() assert (response.status_code in [200, 401, 400, 404])
  16. class EventEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, HistoricalEvent): return {

    "id": obj.id "description": obj.description "event_time":obj.event_time.isoformat() } return json.JSONEncoder.default(self, obj)
  17. @given(st.integers(), st.text(), datetimes(timezones=['UTC'])) def test_to_from_json(id, desc, date): original_event = HistoricalEvent(id,

    desc, date) encoded_event = json.dumps(event, cls=EventEncoder) decoded_event = json.loads( encoded_event, object_hook=HistoricalEvent.fromJson ) assert decoded_event == original_event
  18. @given(st.integers(), st.text(), datetimes(timezones=['UTC'])) def test_to_from_json(id, desc, date): original_event = HistoricalEvent(id,

    desc, date) encoded_event = json.dumps(event, cls=EventEncoder) decoded_event = json.loads( encoded_event, object_hook=HistoricalEvent.fromJson ) assert decoded_event == original_event
  19. @given(st.integers(), st.text(), datetimes(timezones=['UTC'])) def test_to_from_json(id, desc, date): original_event = HistoricalEvent(id,

    desc, date) encoded_event = json.dumps(event, cls=EventEncoder) decoded_event = json.loads( encoded_event, object_hook=HistoricalEvent.fromJson ) assert decoded_event == original_event
  20. @given(st.integers(), st.text(), datetimes(timezones=['UTC'])) def test_to_from_json(id, desc, date): original_event = HistoricalEvent(id,

    desc, date) encoded_event = json.dumps(event, cls=EventEncoder) decoded_event = json.loads( encoded_event, object_hook=HistoricalEvent.fromJson ) assert decoded_event == original_event
  21. @given(st.integers(), st.text(), datetimes(timezones=['UTC'])) def test_to_from_json(id, desc, date): original_event = HistoricalEvent(id,

    desc, date) encoded_event = json.dumps(event, cls=EventEncoder) decoded_event = json.loads( encoded_event, object_hook=HistoricalEvent.fromJson ) assert decoded_event == original_event
  22. Stateful Tests • Define a state • What operations can

    happen in what conditions? • How do operations affect the state? • What must be true for each step?
  23. class HeapMachine(RuleBasedStateMachine): Heaps = Bundle('heaps') @rule(target=Heaps) def new_heap(self): return Heap()

    @rule(heap=Heaps, value=integers()) def heap_push(self, heap, value): push(heap, value) http://hypothesis.works/articles/rule-based-stateful-testing/
  24. class HeapMachine(RuleBasedStateMachine): Heaps = Bundle('heaps') @rule(target=Heaps) def new_heap(self): return Heap()

    @rule(heap=Heaps, value=integers()) def heap_push(self, heap, value): push(heap, value) http://hypothesis.works/articles/rule-based-stateful-testing/
  25. http://hypothesis.works/articles/rule-based-stateful-testing/ … @rule(target=Heaps, heap1=Heaps, heap2=Heaps) def merge(self, heap1, heap2): return

    heap_merge(heap1, heap2) @rule(heap=Heaps.filter(bool)) def pop(self, heap): correct = max(list(heap)) result = heap_pop(heap) assert correct == result
  26. http://hypothesis.works/articles/rule-based-stateful-testing/ … @rule(target=Heaps, heap1=Heaps, heap2=Heaps) def merge(self, heap1, heap2): return

    heap_merge(heap1, heap2) @rule(heap=Heaps.filter(bool)) def pop(self, heap): correct = max(list(heap)) result = heap_pop(heap) assert correct == result
  27. Property Based Testing • Describe the arguments • Describe the

    result • Have the computer try to prove your code wrong
  28. Your Turn • Download Hypothesis • Use it • Share

    how you used it • FIND MORE PATTERNS
  29. More about Hypothesis • http://hypothesis.works/ More On Property Based Testing

    In General • http://www.quviq.com/ • https://fsharpforfunandprofit.com/posts/property-based-testing-2/ Testing Eventual Consistency with RIAK • https://www.youtube.com/watch?v=x9mW54GJpG0