Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
The Path Towards Simplifying Consistency in Distributed Systems
Search
Caitie McCaffrey
April 24, 2017
Technology
1
230
The Path Towards Simplifying Consistency in Distributed Systems
Caitie McCaffrey
April 24, 2017
Tweet
Share
More Decks by Caitie McCaffrey
See All by Caitie McCaffrey
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
325
21k
Argus Papers We Love
caitiem20
13
1.1k
The Verification of a Distributed System
caitiem20
22
2.1k
We Hear You Like Papers: Eventual Consistency
caitiem20
14
730
The Verification of a Distributed System
caitiem20
12
690
The Verification of a Distributed System
caitiem20
6
710
A Brief History of Distributed Programming: RPC
caitiem20
31
6.4k
Building Scalable Stateful Services
caitiem20
12
1.4k
Papers we Love PDX
caitiem20
2
1.1k
Other Decks in Technology
See All in Technology
ギークの理想が7つ集まるエムスリーで夢を叶えよう - エムスリー株式会社
m3_engineering
1
260
サーバーレスAPI(API Gateway+Lambda)とNext.jsで 個人ブログを作ろう!
shuntaka
PRO
0
560
Flutter研修【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
160
スタートアップにおける組織設計とスクラムの長期戦略 / Scrum Fest Kanazawa 2024
yoshikiiida
13
3.6k
Classmethod流のPlatform Engineering / classmethod-platform-engineering-devio2024
tomoki10
0
470
AWSで”最小権限の原則”を実現するための考え方 /20240722-ssmjp-aws-least-privilege
opelab
10
4.3k
クラウド利用者の「責任」をどう果たす?AWSセキュリティ対策のススメ #AWSSummit
hiashisan
0
270
可視化プラットフォームGrafanaの基本と活用方法の全て
hamadakoji
0
230
推薦システムを本番導入する上で一番優先すべきだったこと~NewsPicks記事推薦機能の改善事例を元に~
morinota
0
120
Azure AI ことはじめ
tsubakimoto_s
0
130
20240717_イケコパ代表Copilot_in_Teams会社でこう使ってます
ponponmikankan
2
430
成長期に歩みを止めないための創業期の開発文化形成
mayah
6
420
Featured
See All Featured
Building a Scalable Design System with Sketch
lauravandoore
458
32k
Making Projects Easy
brettharned
111
5.7k
Into the Great Unknown - MozCon
thekraken
20
1.3k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
155
14k
Optimizing for Happiness
mojombo
373
69k
Bash Introduction
62gerente
607
210k
Product Roadmaps are Hard
iamctodd
PRO
48
10k
How STYLIGHT went responsive
nonsquared
93
5k
Side Projects
sachag
451
42k
Designing for humans not robots
tammielis
247
25k
The Brand Is Dead. Long Live the Brand.
mthomps
52
36k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
78
15k
Transcript
The Path Towards Simplifying Consistency in Distributed Systems
Caitie McCaffrey Distributed Systems Engineer caitiem.com @caitie
Distributed Systems Engineer Feral Concurrency Control Programmer Caitie McCaffrey
2015
2015 “Application-level Mechanisms for maintaining database integrity”
Halo 4 Statistics Service Halo 4 Statistics Service Record &
Aggregate Statistics From Every Multiplayer Halo Game a User Plays
Halo 4 Statistics Service
Observability at Twitter Make Dashboards & Alerts Always Available
Cmd Line Tool Viz / Dashboad Alerting Svc Cuckoo-Read Cuckoo-Write
Indexing Svc Relay Svc Twitter Front End Twitter Svc Twitter Statsite Twitter Svc Twitter Svc Scribe Collection Agent HDFS Manhattan Database Public Cloud Observability at Twitter
Notify Users When We Receive Abuse Reports
Notify Users If An Action is Taken
Report Events Stream Modification Events Stream Enrollment Events Stream Listener
Service Daemons Service Executor Service Notification Events Stream Notification Service Manhattan Macaw
Feral Concurrency Control
Service Service Service Monoliths
The Rise of NoSQL
None
Microservices
State of Affairs
Linearizable Sequential Causal Pipelined Random Access Memory Read Your Write
Monotonic Read Monotonic Write Write From Read Consistency Models
None
None
None
None
None
None
Can We Do Better?
Finance & Accounting
VisiCalc 1979
1983 Lotus 123
None
1987 Excel
None
& Standardization Computer Aided Computation
Standardization for Consistency in Distributed Systems
1988
1988 “We believe the form of communication that is needed
is remote procedure call with at- most-once semantics”
1988 “An adequate language must provide a modular, reasonably automatic
method for achieving consistency ”
Branch B Account 789: $250 Branch A Account 123 :
$100 Account 456: $50 SubAction: Deposit (123, 50) Transfer Action SubAction: Withdraw(789, 50) Transfer Deposit Withdraw enter topaction coenter action branchA.Deposit(123, 50) action branchB.Withdraw(789, 50) end end
What about Spanner?
2015
2015 “The biggest barrier…is that consistency mechanisms must integrate across
many stateful services”
Microservices
CRDTs * Stolen from Chris Meiklejohn in practice
CRDTs * Stolen from Chris Meiklejohn in practice
There Is No One Size Fits All Consistency Model
Mixed Consistency Data Stores Manhattan
None
Computer Aided Computation for Consistency in Distributed Systems
2016
Bank Application Bank Account must be > 0 Deposit Money
Withdrawal Money
‘Cause I’m Strong Enough: Reasoning About Consistency Choices in Distributed
Systems
Define Invariants & Actions Deposit Money Withdrawal Money Determine Consistency
Required Bank Account > 0 Eventual Consistency Strong Consistency Code Generation?
& Standardization Computer Aided Computation
Thank you @caitie