Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Property-based Testing is a Mindset
Search
Andrea Leopardi
December 06, 2017
Programming
0
690
Property-based Testing is a Mindset
Andrea Leopardi
December 06, 2017
Tweet
Share
More Decks by Andrea Leopardi
See All by Andrea Leopardi
gen_statem - OTP's Unsung Hero
whatyouhide
2
250
The World is a Network (and We Are Just Nodes)
whatyouhide
1
220
BEAM: The Perfect Fit for Networks
whatyouhide
1
200
Update from the Elixir team - 2022
whatyouhide
0
400
Testing Asynchronous OTP
whatyouhide
1
520
Elixir Sightseeing Tour
whatyouhide
0
420
Mint - Disrupting HTTP clients
whatyouhide
0
250
BEAM Architecture Handbook
whatyouhide
7
2.8k
The Evolution of a Language
whatyouhide
0
160
Other Decks in Programming
See All in Programming
Benchmark
sysong
0
250
Webの外へ飛び出せ NativePHPが切り拓くPHPの未来
takuyakatsusa
2
340
datadog dash 2025 LLM observability for reliability and stability
ivry_presentationmaterials
0
110
すべてのコンテキストを、 ユーザー価値に変える
applism118
2
750
Cursor AI Agentと伴走する アプリケーションの高速リプレイス
daisuketakeda
1
130
来たるべき 8.0 に備えて React 19 新機能と React Router 固有機能の取捨選択とすり合わせを考える
oukayuka
2
850
PHPでWebSocketサーバーを実装しよう2025
kubotak
0
100
Team operations that are not burdened by SRE
kazatohiei
1
190
CursorはMCPを使った方が良いぞ
taigakono
1
170
XP, Testing and ninja testing
m_seki
3
180
What Spring Developers Should Know About Jakarta EE
ivargrimstad
0
220
ドメインモデリングにおける抽象の役割、tagless-finalによるDSL構築、そして型安全な最適化
knih
11
2k
Featured
See All Featured
The Pragmatic Product Professional
lauravandoore
35
6.7k
Unsuck your backbone
ammeep
671
58k
Building Adaptive Systems
keathley
43
2.6k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.8k
Designing Experiences People Love
moore
142
24k
Statistics for Hackers
jakevdp
799
220k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
490
Practical Orchestrator
shlominoach
188
11k
Six Lessons from altMBA
skipperchong
28
3.8k
Rebuilding a faster, lazier Slack
samanthasiow
81
9.1k
Transcript
I S A M I N D S E T
P R O P E R T Y - B A S E D T E S T I N G
@whatyouhide
None
Elixir
T E S T I N G
why do we test? no tests yes tests
unit tests
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end example-based
table-based Input Output [] [] [1, 2, 3] [1, 2,
3] [2, 1, 3] [1, 2, 3]
unit tests , but also
P R O P E R T I E S
hard to write , but...
valid inputs properties of output testing framework
github.com/whatyouhide/stream_data
example time: sorting lists
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end
lists of integers
it's a list has the same elements it's ordered
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
def sort(list), do: list
[32, 2, 44, -12] [1, 0] *shrinking
P A T T E R N S
circular code
decode(encode(term)) == term
property "encoding->decoding is circular" do check all bin <- binary()
do encoded = Huffman.encode(bin) assert is_binary(encoded) assert Huffman.decode(encoded) == bin end end Huffman encoding
oracle model
my_code() == oracle_code()
older system less performant implementation
property "gives same results as Erlang impl" do check all
bin <- binary() do assert Huffman.encode(bin) == :huffman.encode(bin) end end
smoke tests https://www.youtube.com/watch?v=jvwfDdgg93E
API: 200, 201, 400, 404 https://www.youtube.com/watch?v=jvwfDdgg93E
https://www.youtube.com/watch?v=jvwfDdgg93E property "only expected codes are returned" do check all
request <- request() do response = HTTP.perform(request) assert response.status in [200, 201, 400, 404] end end
locally , CI
if System.get_env("CI") == "true" do config :stream_data, max_runs: 500 else
config :stream_data, max_runs: 25 end
unit + properties
R E A L - W O R L D
E X A M P L E
Redis parser
continuation parser parse(bytes) {:ok, command} {:more, function}
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
S T A T E F U L T E
S T I N G
model valid commands +
model: state + state transformations
commands: calls + preconditions
bounded-size queue
Max size model system
•push(queue, value) •resize(queue, max_size) commands
def push(model, _value), do: model def resize(_model, max_size), do: max_size
push(queue, {:some, "value"}) push(queue, [123, 3.2]) resize(queue, 1) push(queue, %{})
assert size(queue) <= model
LevelDB
17 (seventeen) calls 33 (thirty three) calls
C O N C L U S I O N
find obscure bugs reduce to minimal failing input find specification
errors keep increasing input space
v1.7
use stream_data
use property-based testing
@whatyouhide github.com/whatyouhide/stream_data