Mutation Testing

Have you ever wanted to testing super powers? Well mutations could be your ticket!

In this talk we'll be looking at how to check your unit tests rock or suck. We'll be using mutation testing to test your test. A must know for anyone who comes into contact with unit tests.

Transcript

1. armakuni.com Mutation Testing Berlin PHP UG Billie Thompson
2. About me 2 • Consulting Engineer for Armakuni • I help “Return creativity & joy to the world of software engineering” • Quite tall • Favourite animal is the capybara
3. We’ve worked with 3
4. What is it? Break your code Ensure a test fails Improve test 4
5. Warm-up
6. Do you all know your TDD Cycle Red Green Refactor Red 6
7. Todo list • 7 • TDD • Demo: When the test doesn’t fail • When it’s useful • Demo: Infection
8. TDD 8
9. Red 9
10. Green 10
11. Refactor 11
12. Red 12
13. Demo: Break It
14. What happens if the tests pass? 14
15. Bowling 15
16. Bowling 16
17. 10 Pins 17
18. 10 sets of up to 2 rolls 18
19. A score card 1 2 3 4 5 6 7 8 9 10 19
20. Rules: Standard Set 1 Set 2 Roll 1 Roll 2 Roll 1 Roll 2 Pins 1 1 1 1 Score 1 2 3 4 20
21. Rules: Spare Set 1 Set 2 Roll 1 Roll 2 Roll 1 Roll 2 Pins 5 5 1 1 Score 5 10 12 13 21
22. Rules: Strike Set 1 Set 2 Roll 1 Roll 2 Roll 1 Roll 2 Pins 10 1 1 Score 10 12 14 22
23. Demo: Break It
25. What is it? Break your code Ensure a test fails Improve test 25
26. Step by Step Build AST Introduce Bug Do Tests Fail? 26
27. Step by Step Build AST Introduce Bug Do Tests Fail? Introduce Bug Do Tests Fail? Introduce Bug Do Tests Fail? 27
28. 28 Infection has 134 ways to break your code
29. Todo list • 29 ✓ TDD ✓ Demo: When the test doesn’t fail • When it’s useful • Demo: Infection
30. Stefan has inherited a codebase 30
31. What bad habit does Stefan have? 31
32. Bugs keep appearing 32
33. Stefan gets stricter with reviews 33
34. Find untested code 34
35. Other Use Cases • 35 • Open Source Projects • New languages • Improving your heuristic toolkit
36. Todo list • 36 ✓ TDD ✓ Demo: When the test doesn’t fail ✓ When it’s useful • Demo: Infection
37. Demo: Infection
38. armakuni.com Billie Thompson Thank you! • https://github.com/armakuni/bad-bowling-kata • https://infection.github.io/ @PurpleBooth @ArmakuniHQ