Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
CRDTs - The science behind Phoenix Presence
Search
Maciej Kaszubowski
May 25, 2017
Programming
2
260
CRDTs - The science behind Phoenix Presence
Maciej Kaszubowski
May 25, 2017
Tweet
Share
More Decks by Maciej Kaszubowski
See All by Maciej Kaszubowski
Error-free Elixir
mkaszubowski
0
330
Modular Design in Elixir (ElixirConf EU 2019)
mkaszubowski
2
720
The Big Ball of Nouns
mkaszubowski
0
97
Modular Design in Elixir
mkaszubowski
1
380
Our three years with Elixir
mkaszubowski
0
240
Concurrency Basics for Elixir
mkaszubowski
0
120
Distributed Elixir
mkaszubowski
0
150
Software Architecture
mkaszubowski
0
130
Let it crash - fault tolerance in Elixir/OTP
mkaszubowski
0
470
Other Decks in Programming
See All in Programming
PHPで始める振る舞い駆動開発(Behaviour-Driven Development)
ohmori_yusuke
2
410
“いい感じ“な定量評価を求めて - Four Keysとアウトカムの間の探求 -
nealle
2
11k
20250704_教育事業におけるアジャイルなデータ基盤構築
hanon52_
5
880
Rails Frontend Evolution: It Was a Setup All Along
skryukov
0
220
レベル1の開発生産性向上に取り組む − 日々の作業の効率化・自動化を通じた改善活動
kesoji
0
260
#kanrk08 / 公開版 PicoRubyとマイコンでの自作トレーニング計測装置を用いたワークアウトの理想と現実
bash0c7
1
870
Startups on Rails in Past, Present and Future–Irina Nazarova, RailsConf 2025
irinanazarova
0
150
オンコール⼊⾨〜ページャーが鳴る前に、あなたが備えられること〜 / Before The Pager Rings
yktakaha4
1
550
ISUCON研修おかわり会 講義スライド
arfes0e2b3c
1
460
Goで作る、開発・CI環境
sin392
0
240
明示と暗黙 ー PHPとGoの インターフェイスの違いを知る
shimabox
2
580
「テストは愚直&&網羅的に書くほどよい」という誤解 / Test Smarter, Not Harder
munetoshi
0
190
Featured
See All Featured
Product Roadmaps are Hard
iamctodd
PRO
54
11k
The Pragmatic Product Professional
lauravandoore
35
6.7k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
The Cult of Friendly URLs
andyhume
79
6.5k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
45
7.5k
The Cost Of JavaScript in 2023
addyosmani
51
8.5k
Writing Fast Ruby
sferik
628
62k
Documentation Writing (for coders)
carmenintech
72
4.9k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Transcript
The Problem
None
Server Node 1 Server Node 2
Node A Node B [User] [User] [User] [] [] [User]
User connects User disconnects
There's no global time
Node A Node B [User] [User] [User] [] [] [User]
User connects User disconnects
Node A Node B [User] [User] [User] [] [] [User]
User connects User disconnects
Node A Node B
P.track(self, "users", "U1", %{})
P.track(self, "users", "U1", %{}) P.list("users") %{"U1" %{metas: [%{phx_ref: "…"}]}}
P.track(self, "users", "U1", %{}) P.list("users") P.track(self, "users", "U2", %{}) %{"U1"
%{metas: [%{phx_ref: "…"}]}}
P.track(self, "users", "U1", %{}) P.list("users") P.track(self, "users", "U2", %{}) P.list("users")
%{"U1" %{metas: [%{phx_ref: "…"}]}, "U2" %{metas: [%{phx_ref: "…"}]}} %{"U1" %{metas: [%{phx_ref: "…"}]}}
P.track(self, "users", "U1", %{}) P.list("users") P.track(self, "users", "U2", %{}) P.list("users")
%{"U1" %{metas: [%{phx_ref: "…"}]}, "U2" %{metas: [%{phx_ref: "…"}]}} P.track(self, "users", "U1", %{}) P.untrack(self, "users", "U1") %{"U1" %{metas: [%{phx_ref: "…"}]}}
P.track(self, "users", "U1", %{}) P.list("users") P.track(self, "users", "U2", %{}) P.list("users")
%{"U1" %{metas: [%{phx_ref: "…"}]}, "U2" %{metas: [%{phx_ref: "…"}]}} P.track(self, "users", "U1", %{}) P.untrack(self, "users", "U1") P.list("users") %{"U1" %{metas: [%{phx_ref: "…"}]}, "U2" %{metas: [%{phx_ref: "…"}]}} P.list("users") %{"U1" %{metas: [%{phx_ref: "…"}]}, "U2" %{metas: [%{phx_ref: "…"}]}} %{"U1" %{metas: [%{phx_ref: "…"}]}}
CRDTs The science behind Phoenix Presence Maciej Kaszubowski
Conflict-free Replicated Data Type
Alternatives
• Single source of truth (DB) • Consensus algorithm •
Resolving conflicts manually
Why CRDTs?
Eventually consistent Highly available Easy to use
Eventually consistent Highly available Easy to use Hard to create
:(
Features
1. Commutative 2. Associative 3. Idempotent x y = y
x (x y) z = x (y z) x x = x
Server Node 1 Server Node 2 Server Node 3
Client Client Client Client Client Client Client Client Client Server
Examples
Counters
Node A Node B +5 5 8 -2 3 +5
User connects User disconnects 0 0 5 3 8
Node A Node B +5 5 8 -2 8 +5
User connects User disconnects 0 0 5 3 13 10
G-Counter Grow-only counter
Node 2: 3 Value=8 Node 1: 5 Node 3: 1
Merge
Node 2: 3 Value=9 Node 1: 6 Node 3: 1
+1 Merge
PN-Counter Positive-Negative Counter
Node 2: P=2 N=2 Value=7 Node 1: P=5 N=2 Node
3: P=4 N=0 3 0 4 Merge
Node 2: P=2 N=2 Value=6 Node 1: P=5 N=3 Node
3: P=4 N=0 2 0 4 +1 Merge
Node 2: P=2 N=2 Value=8 Node 1: P=5 N=3 Node
3: P=6 N=0 2 0 6 +2 +1 Merge
Sets
Node A Node B [User] [User] [User] [] [] [User]
User connects User disconnects
G-Set Grow-only set
Node 2: [1,2] Value=[1,2,3,4] Node 1: [1] Node 3: [3,4]
Merge
Node 2: [1,2] Value=[1,2,3,4,5] Node 1: [1,5] Node 3: [3,4]
Merge
2P-Set Two-phase set
Node 2: [1,2],[] Value=[1,2,3,4] Node 1: [1],[] Node 3: [3,4],[]
Merge
Node 2: [1,2],[] Value=[1,2,3,4] Node 1: [1],[] Node 3: [3,4],[]
Merge G-Set for adds
Node 2: [1,2],[] Value=[1,2,4] Node 1: [1],[] Node 3: [3,4],[3]
Merge G-Set for removals
Elements cannot be re-added
Removes win
Node A Node B [ ] Add Remove [1] [
] [1]
Node A Node B [ ] [1] Add Remove [1]
[1] [ ] [1] [ ] [1]
Node A Node B [ ] [1] Add Remove [1]
[1] [ ] [1] [ ] [1]
Node A Node B [ ] [1] [1] Add Remove
[1] [1] [1] [ ] [1] [1] [1] [ ] [1]
OR-Set Observed-remove set
Node A Node B [ ] Add Remove [{A,1}] [
] [{A,1}]
Node A Node B [ ] [ ] Add Remove
[{A,1}] [{A,1}] [ ] [ ] [{A,1}] [{A,1}, ]
Node A Node B [ ] [ 1 ] Add
Remove [{A,1}] [{A,1}] [ ] [ ] [{A,1}] [{A,1},{A,2}]
Node A Node B [ ] [ 1 ] Add
Remove [{A,1}] [{A,1}] [ ] [ ] [{A,1}] [{A,1},{A,2}]
Node A Node B [ ] [ 1 ] Add
Remove [{A,1}] [{A,1}] [ ] [ ] [{A,1}] [{A,1},{A,2}] [ 1 ] [{A,1},{A,2}] [{A,1},{A,2}] [ 1 ]
Node A Node B [ ] [ 1 ] Add
Remove [{A,1}] [{A,1}] [ ] [ ] [{A,1}] [{A,1},{A,2}] [ 1 ] [{A,1},{A,2}] [{A,1},{A,2}] [ 1 ] [A] [A]
Add 1000 Remove 1000 Add Remove …
ORSWOT Observed-remove set without tombstones
None
None
Use Cases
None
None
None
None
• Load balancing / routing • Mobile clients synchronisation •
Temporary data on the servers • Avoiding work duplication • Collaborative editing
None
lasp-lang.readme.io
Summary
You're (almost) always designing a distributed system
Think about failures
Choose the correct tool for the job
References • https://medium.com/@istanbul_techie/a-look-at-conflict-free-replicated-data- types-crdt-221a5f629e7e • http://basho.com/posts/technical/distributed-data-types-riak-2-0/ • http://highscalability.com/blog/2014/10/13/how-league-of-legends-scaled- chat-to-70-million-players-it-t.html •
https://hal.inria.fr/inria-00609399v1/document • https://developers.soundcloud.com/blog/roshi-a-crdt-system-for- timestamped-events
Thanks!