Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Eeek, my tests are mutating
Search
Lander Vanderstraeten
November 07, 2017
Programming
1
94
Eeek, my tests are mutating
Presentation given for
https://www.meetup.com/phpgent/events/242279704
Lander Vanderstraeten
November 07, 2017
Tweet
Share
Other Decks in Programming
See All in Programming
Jakarta EE Core Profile and Helidon - Speed, Simplicity, and AI Integration
ivargrimstad
0
310
オープンセミナー2025@広島「君はどこで動かすか?」アンケート結果
satoshi256kbyte
0
230
print("Hello, World")
eddie
1
370
旅行プランAIエージェント開発の裏側
ippo012
1
610
MCPとデザインシステムに立脚したデザインと実装の融合
yukukotani
3
1.1k
testingを眺める
matumoto
1
130
CSC305 Summer Lecture 12
javiergs
PRO
0
130
2025 年のコーディングエージェントの現在地とエンジニアの仕事の変化について
azukiazusa1
15
6.7k
Kiroで始めるAI-DLC
kaonash
2
520
速いWebフレームワークを作る
yusukebe
4
1.6k
Protocol Buffersの型を超えて拡張性を得る / Beyond Protocol Buffers Types Achieving Extensibility
linyows
0
100
Improving my own Ruby thereafter
sisshiki1969
1
150
Featured
See All Featured
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
For a Future-Friendly Web
brad_frost
179
9.9k
How STYLIGHT went responsive
nonsquared
100
5.8k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
GraphQLとの向き合い方2022年版
quramy
49
14k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
800
How to train your dragon (web standard)
notwaldorf
96
6.2k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
1.5k
Transcript
HI, I'M LANDER VANDERSTRAETEN 1
3 QUIZ QUESTIONS 2
WHO THINKS UNIT TESTS ARE WASTE OF TIME? 3
4
WHO ALWAYS WRITES UNIT TESTS? 5
6
WHO ALREADY USES MUTATION TESTING? 7
8
CODE HAS BUGS TESTS ARE CODE TESTS HAVE BUGS! 9
10
TESTS ARE A GOOD TOOL TO VERIFY YOUR CODE BUT
HOW DO YOU VERIFY YOUR TESTS? 11
CODE COVERAGE IS A START, BUT IT CAN GIVE A
“GOOD” SCORE WITH USELESS TESTS 12
ALL TESTS PASSED! 13
HOW WE DETECT? 14
MUTATION TESTING 15
Mutation testing is a technique to evaluate the quality of
your tests by programmatically mutating and making a series of small modifications to your code so that the tests no longer pass. 16
17
IF BUGS ARE CRIMES AND YOUR TESTS ARE THE POLICE
MUTATIONS ARE FAKE CRIMES THAT LET YOU SEE THE POLICE IS DOING THEIR JOB 18
TO ASSESS THE QUALITY OF A GIVEN TEST, MUTANTS ARE
EXECUTED AGAINST THE INPUT TEST TO SEE IF THE SEEDED FAULTS CAN BE DETECTED. 19
MUTATION, MUTANTS, MUTAFUCK WHAT? 20
Each mutated version: Mutant Mutated program + failing tests: killed
mutant Mutated program + passed tests: escaped mutant 21
MEASURE ALL THE THINGS! 22
Test suites are measured by the percentage of mutants that
they kill. New tests can be designed to kill additional mutants. 23
HOW DOES IT MODIFY? 24
MUTATORS 25
EXAMPLE public function foobar(int $number): int { return $number +
1; // original } public function foobar(int $number): int { return $number - 1; // mutated version } 26
ANOTHER EXAMPLE public function foobar(?Bar $foo): string { if (null
=== $foo) { // original return 'foo'; } // ... } public function foobar(?Bar $foo): string { if (null !== $foo) { // mutated version return 'foo'; } // ... } 27
DEMO TIME WITH INFECTION 28
WHAT DID WE LEARN TODAY? 29
1. Always write unit tests 2. Code coverage gives a
false positive feeling 3. Mutation testing also gives a false positive feeling 30
QUESTIONS? 31