Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Eeek, my tests are mutating
Search
Lander Vanderstraeten
November 07, 2017
Programming
1
94
Eeek, my tests are mutating
Presentation given for
https://www.meetup.com/phpgent/events/242279704
Lander Vanderstraeten
November 07, 2017
Tweet
Share
Other Decks in Programming
See All in Programming
MCPでVibe Working。そして、結局はContext Eng(略)/ Working with Vibe on MCP And Context Eng
rkaga
5
2.3k
Ruby Parser progress report 2025
yui_knk
1
450
請來的 AI Agent 同事們在寫程式時,怎麼用 pytest 去除各種幻想與盲點
keitheis
0
120
そのAPI、誰のため? Androidライブラリ設計における利用者目線の実践テクニック
mkeeda
2
1.8k
Ruby×iOSアプリ開発 ~共に歩んだエコシステムの物語~
temoki
0
340
AI Coding Agentのセキュリティリスク:PRの自己承認とメルカリの対策
s3h
0
230
プロポーザル駆動学習 / Proposal-Driven Learning
mackey0225
2
1.3k
Namespace and Its Future
tagomoris
6
710
ProxyによるWindow間RPC機構の構築
syumai
3
1.2k
Navigating Dependency Injection with Metro
zacsweers
3
2.5k
Azure SRE Agentで運用は楽になるのか?
kkamegawa
0
2.4k
testingを眺める
matumoto
1
140
Featured
See All Featured
Building Better People: How to give real-time feedback that sticks.
wjessup
368
19k
Git: the NoSQL Database
bkeepers
PRO
431
66k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.5k
A designer walks into a library…
pauljervisheath
207
24k
GraphQLの誤解/rethinking-graphql
sonatard
72
11k
We Have a Design System, Now What?
morganepeng
53
7.8k
The Straight Up "How To Draw Better" Workshop
denniskardys
236
140k
Visualization
eitanlees
148
16k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.1k
Practical Orchestrator
shlominoach
190
11k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
Transcript
HI, I'M LANDER VANDERSTRAETEN 1
3 QUIZ QUESTIONS 2
WHO THINKS UNIT TESTS ARE WASTE OF TIME? 3
4
WHO ALWAYS WRITES UNIT TESTS? 5
6
WHO ALREADY USES MUTATION TESTING? 7
8
CODE HAS BUGS TESTS ARE CODE TESTS HAVE BUGS! 9
10
TESTS ARE A GOOD TOOL TO VERIFY YOUR CODE BUT
HOW DO YOU VERIFY YOUR TESTS? 11
CODE COVERAGE IS A START, BUT IT CAN GIVE A
“GOOD” SCORE WITH USELESS TESTS 12
ALL TESTS PASSED! 13
HOW WE DETECT? 14
MUTATION TESTING 15
Mutation testing is a technique to evaluate the quality of
your tests by programmatically mutating and making a series of small modifications to your code so that the tests no longer pass. 16
17
IF BUGS ARE CRIMES AND YOUR TESTS ARE THE POLICE
MUTATIONS ARE FAKE CRIMES THAT LET YOU SEE THE POLICE IS DOING THEIR JOB 18
TO ASSESS THE QUALITY OF A GIVEN TEST, MUTANTS ARE
EXECUTED AGAINST THE INPUT TEST TO SEE IF THE SEEDED FAULTS CAN BE DETECTED. 19
MUTATION, MUTANTS, MUTAFUCK WHAT? 20
Each mutated version: Mutant Mutated program + failing tests: killed
mutant Mutated program + passed tests: escaped mutant 21
MEASURE ALL THE THINGS! 22
Test suites are measured by the percentage of mutants that
they kill. New tests can be designed to kill additional mutants. 23
HOW DOES IT MODIFY? 24
MUTATORS 25
EXAMPLE public function foobar(int $number): int { return $number +
1; // original } public function foobar(int $number): int { return $number - 1; // mutated version } 26
ANOTHER EXAMPLE public function foobar(?Bar $foo): string { if (null
=== $foo) { // original return 'foo'; } // ... } public function foobar(?Bar $foo): string { if (null !== $foo) { // mutated version return 'foo'; } // ... } 27
DEMO TIME WITH INFECTION 28
WHAT DID WE LEARN TODAY? 29
1. Always write unit tests 2. Code coverage gives a
false positive feeling 3. Mutation testing also gives a false positive feeling 30
QUESTIONS? 31