Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Property-based Testing is a Mindset
Search
Andrea Leopardi
December 06, 2017
Programming
0
690
Property-based Testing is a Mindset
Andrea Leopardi
December 06, 2017
Tweet
Share
More Decks by Andrea Leopardi
See All by Andrea Leopardi
gen_statem - OTP's Unsung Hero
whatyouhide
2
250
The World is a Network (and We Are Just Nodes)
whatyouhide
1
220
BEAM: The Perfect Fit for Networks
whatyouhide
1
200
Update from the Elixir team - 2022
whatyouhide
0
410
Testing Asynchronous OTP
whatyouhide
1
530
Elixir Sightseeing Tour
whatyouhide
0
430
Mint - Disrupting HTTP clients
whatyouhide
0
260
BEAM Architecture Handbook
whatyouhide
7
2.8k
The Evolution of a Language
whatyouhide
0
160
Other Decks in Programming
See All in Programming
兎に角、コードレビュー
mitohato14
0
130
Reactの歴史を振り返る
tutinoko
1
180
技術的負債で信頼性が限界だったWordPress運用をShifterで完全復活させた話
rvirus0817
1
1.8k
Claude Code と OpenAI o3 で メタデータ情報を作る
laket
0
130
Understanding Kotlin Multiplatform
l2hyunwoo
0
260
あのころの iPod を どうにか再生させたい
orumin
2
2.5k
LLMは麻雀を知らなすぎるから俺が教育してやる
po3rin
3
2.1k
自作OSでDOOMを動かしてみた
zakki0925224
1
1.4k
20250808_AIAgent勉強会_ClaudeCodeデータ分析の実運用〜競馬を題材に回収率100%の先を目指すメソッドとは〜
kkakeru
0
180
あなたとJIT, 今すぐアセンブ ル
sisshiki1969
1
690
DockerからECSへ 〜 AWSの海に出る前に知っておきたいこと 〜
ota1022
5
1.4k
可変性を制する設計: 構造と振る舞いから考える概念モデリングとその実装
a_suenami
10
1.8k
Featured
See All Featured
Git: the NoSQL Database
bkeepers
PRO
431
65k
KATA
mclloyd
32
14k
The Language of Interfaces
destraynor
160
25k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
1k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
Designing for Performance
lara
610
69k
It's Worth the Effort
3n
186
28k
Producing Creativity
orderedlist
PRO
347
40k
Six Lessons from altMBA
skipperchong
28
4k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
A better future with KSS
kneath
239
17k
Transcript
I S A M I N D S E T
P R O P E R T Y - B A S E D T E S T I N G
@whatyouhide
None
Elixir
T E S T I N G
why do we test? no tests yes tests
unit tests
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end example-based
table-based Input Output [] [] [1, 2, 3] [1, 2,
3] [2, 1, 3] [1, 2, 3]
unit tests , but also
P R O P E R T I E S
hard to write , but...
valid inputs properties of output testing framework
github.com/whatyouhide/stream_data
example time: sorting lists
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end
lists of integers
it's a list has the same elements it's ordered
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
def sort(list), do: list
[32, 2, 44, -12] [1, 0] *shrinking
P A T T E R N S
circular code
decode(encode(term)) == term
property "encoding->decoding is circular" do check all bin <- binary()
do encoded = Huffman.encode(bin) assert is_binary(encoded) assert Huffman.decode(encoded) == bin end end Huffman encoding
oracle model
my_code() == oracle_code()
older system less performant implementation
property "gives same results as Erlang impl" do check all
bin <- binary() do assert Huffman.encode(bin) == :huffman.encode(bin) end end
smoke tests https://www.youtube.com/watch?v=jvwfDdgg93E
API: 200, 201, 400, 404 https://www.youtube.com/watch?v=jvwfDdgg93E
https://www.youtube.com/watch?v=jvwfDdgg93E property "only expected codes are returned" do check all
request <- request() do response = HTTP.perform(request) assert response.status in [200, 201, 400, 404] end end
locally , CI
if System.get_env("CI") == "true" do config :stream_data, max_runs: 500 else
config :stream_data, max_runs: 25 end
unit + properties
R E A L - W O R L D
E X A M P L E
Redis parser
continuation parser parse(bytes) {:ok, command} {:more, function}
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
S T A T E F U L T E
S T I N G
model valid commands +
model: state + state transformations
commands: calls + preconditions
bounded-size queue
Max size model system
•push(queue, value) •resize(queue, max_size) commands
def push(model, _value), do: model def resize(_model, max_size), do: max_size
push(queue, {:some, "value"}) push(queue, [123, 3.2]) resize(queue, 1) push(queue, %{})
assert size(queue) <= model
LevelDB
17 (seventeen) calls 33 (thirty three) calls
C O N C L U S I O N
find obscure bugs reduce to minimal failing input find specification
errors keep increasing input space
v1.7
use stream_data
use property-based testing
@whatyouhide github.com/whatyouhide/stream_data