Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Property-based Testing is a Mindset
Search
Andrea Leopardi
December 06, 2017
Programming
0
690
Property-based Testing is a Mindset
Andrea Leopardi
December 06, 2017
Tweet
Share
More Decks by Andrea Leopardi
See All by Andrea Leopardi
gen_statem - OTP's Unsung Hero
whatyouhide
2
250
The World is a Network (and We Are Just Nodes)
whatyouhide
1
220
BEAM: The Perfect Fit for Networks
whatyouhide
1
200
Update from the Elixir team - 2022
whatyouhide
0
410
Testing Asynchronous OTP
whatyouhide
1
520
Elixir Sightseeing Tour
whatyouhide
0
430
Mint - Disrupting HTTP clients
whatyouhide
0
250
BEAM Architecture Handbook
whatyouhide
7
2.8k
The Evolution of a Language
whatyouhide
0
160
Other Decks in Programming
See All in Programming
Gemini CLIの"強み"を知る! Gemini CLIとClaude Codeを比較してみた!
kotahisafuru
2
710
Jakarta EE Meets AI
ivargrimstad
0
470
副作用と戦う PHP リファクタリング ─ ドメインイベントでビジネスロジックを解きほぐす
kajitack
3
500
Android 15以上でPDFのテキスト検索を爆速開発!
tonionagauzzi
0
180
Vibe Codingの幻想を超えて-生成AIを現場で使えるようにするまでの泥臭い話.ai
fumiyakume
20
9.7k
React は次の10年を生き残れるか:3つのトレンドから考える
oukayuka
41
16k
Vibe coding コードレビュー
kinopeee
0
360
코딩 에이전트 체크리스트: Claude Code ver.
nacyot
0
1k
JetBrainsのAI機能の紹介 #jjug
yusuke
0
160
顧客の画像データをテラバイト単位で配信する 画像サーバを WebP にした際に起こった課題と その対応策 ~継続的な取り組みを添えて~
takutakahashi
4
1.4k
オホーツクでコミュニティを立ち上げた理由―地方出身プログラマの挑戦 / TechRAMEN 2025 Conference
lemonade_37
1
380
GPUを計算資源として使おう!
primenumber
1
300
Featured
See All Featured
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.8k
Code Reviewing Like a Champion
maltzj
524
40k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
We Have a Design System, Now What?
morganepeng
53
7.7k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.5k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Done Done
chrislema
185
16k
A better future with KSS
kneath
238
17k
Faster Mobile Websites
deanohume
308
31k
Thoughts on Productivity
jonyablonski
69
4.8k
Transcript
I S A M I N D S E T
P R O P E R T Y - B A S E D T E S T I N G
@whatyouhide
None
Elixir
T E S T I N G
why do we test? no tests yes tests
unit tests
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end example-based
table-based Input Output [] [] [1, 2, 3] [1, 2,
3] [2, 1, 3] [1, 2, 3]
unit tests , but also
P R O P E R T I E S
hard to write , but...
valid inputs properties of output testing framework
github.com/whatyouhide/stream_data
example time: sorting lists
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end
lists of integers
it's a list has the same elements it's ordered
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
def sort(list), do: list
[32, 2, 44, -12] [1, 0] *shrinking
P A T T E R N S
circular code
decode(encode(term)) == term
property "encoding->decoding is circular" do check all bin <- binary()
do encoded = Huffman.encode(bin) assert is_binary(encoded) assert Huffman.decode(encoded) == bin end end Huffman encoding
oracle model
my_code() == oracle_code()
older system less performant implementation
property "gives same results as Erlang impl" do check all
bin <- binary() do assert Huffman.encode(bin) == :huffman.encode(bin) end end
smoke tests https://www.youtube.com/watch?v=jvwfDdgg93E
API: 200, 201, 400, 404 https://www.youtube.com/watch?v=jvwfDdgg93E
https://www.youtube.com/watch?v=jvwfDdgg93E property "only expected codes are returned" do check all
request <- request() do response = HTTP.perform(request) assert response.status in [200, 201, 400, 404] end end
locally , CI
if System.get_env("CI") == "true" do config :stream_data, max_runs: 500 else
config :stream_data, max_runs: 25 end
unit + properties
R E A L - W O R L D
E X A M P L E
Redis parser
continuation parser parse(bytes) {:ok, command} {:more, function}
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
S T A T E F U L T E
S T I N G
model valid commands +
model: state + state transformations
commands: calls + preconditions
bounded-size queue
Max size model system
•push(queue, value) •resize(queue, max_size) commands
def push(model, _value), do: model def resize(_model, max_size), do: max_size
push(queue, {:some, "value"}) push(queue, [123, 3.2]) resize(queue, 1) push(queue, %{})
assert size(queue) <= model
LevelDB
17 (seventeen) calls 33 (thirty three) calls
C O N C L U S I O N
find obscure bugs reduce to minimal failing input find specification
errors keep increasing input space
v1.7
use stream_data
use property-based testing
@whatyouhide github.com/whatyouhide/stream_data