Consistency for mere mortals - A gentle guide to consistency

This is a kind of foundational slide deck, introducing the most common concepts regarding consistency in a (hopefully) easily digestible way.

It starts with explaining what consistency, constistency models, and anomalies are. Then it dives deeper into consistency models and presents several selected models. Afterwards, some common additional concepts are discussed that are associated with consistency, including a bit of myth-busting. Finally, a few recommendations and a list of the most widespread pitfalls regarding consistency are provided. All this is complemented by a relatively extensive list of references that provide deeper insights.

As always, the voice track is missing (but there will be a recording from J On The Beach 2026 on YouTube). Still, I hope the deck provides a few useful ideas and pointers.

Transcript

1. Consistency for mere mortals A gentle guide to consistency Uwe Friedrichsen – codecentric AG – 2006-2026
2. Uwe Friedrichsen Works @ codecentric https://bsky.app/profile/ufried.bsky.social https://www.speakerdeck.com/ufried https://ufried.com/
3. Consistency?
4. Yikes! Just the thought of it ...
5. What is consistency?
6. Consistency means that changes to some shared data store must work as expected where expected is defined by the consistency model
7. Client Client Node Client Node Node Data store Works as expected Defined by consistency model
8. What is a consistency model?
9. "Informally, a consistency model defines what an application can observe about the updates and reads of its data store” [Sha2018]
10. Client Client Node Client Node Consistency model defines what a client can expect to observe Node Defined by anomalies Data store
11. What is an anomaly?
13. What do we usually expect?
14. We expect strict consistency
15. Strict consistency • Updates are visible everywhere immediately in real-time order • Great model for formalism and thought experiments • Too big availability and latency hit for practical use • Massively restricts concurrent data store access • In practice, weaker consistency models are used • Better balance of consistency, availability, and latency
16. What is an anomaly?
17. An anomaly is any observable deviation from strict consistency
18. Anomalies • Dirty read • Single value scope • T1 updates value (not yet committed) • T2 reads value updated by non-committed T1 • T1 aborts • T2 sees value that officially never existed
19. Anomalies • Dirty read • Non-repeatable read (Fuzzy read) • Single value scope • T1 reads value • T2 updates value and commits • T1 re-reads value and gets different result
20. Anomalies • Dirty read • Non-repeatable read (Fuzzy read) • Phantom read • Multi-value scope • T1 reads value(s) using predicate (i.e., ”WHERE-clause”) • T2 updates value(s) of predicate and commits • T1 re-reads value(s) using predicate and gets different result
21. Anomalies • Dirty read • Non-repeatable read (Fuzzy read) • Phantom read • Read skew • Scope: two values with a constraint (e.g., X+Y > 0) • T1 reads X • T2 updates X and Y and commits • T1 reads Y and observes constraint violation
22. Anomalies • Dirty read • Non-repeatable read (Fuzzy read) • Phantom read • Read skew • Write skew • Scope: two values with a constraint (e.g., X+Y > 0) • T1 and T2 both read X and Y • T1 updates X, T2 updates Y, both commit • Resulting state violates constraint
23. Anomalies • Dirty read • Non-repeatable read (Fuzzy read) • Phantom read • Read skew • Write skew • Lost update • Single value scope • T1 and T2 both update value and commit concurrently • Update of either T1 or T2 is lost
24. Anomalies • Dirty read • Non-repeatable read (Fuzzy read) • Phantom read • Read skew • Write skew • Lost update • Stale read • Single value scope • T1 reads outdated value (that T2 has updated before)
25. Anomalies • Dirty read • Non-repeatable read (Fuzzy read) • Phantom read • Read skew • Write skew • Lost update • Stale read • ...
26. Consistency models
27. Strong consistency (requires coordination) Sticky consistency (requires stable server connection) Weak consistency (coordination-free) Source: https://jepsen.io/consistency/models
28. Consistency model types Coordination Availability Latency Anomalies Strong consistency required o/- o/- +/++ Sticky consistency stable connection + + o Weak consistency none ++ ++ -/--
29. Consistency models – terminology • Single-value vs. multi-value • Single-value consistency models exhibit multi-value anomalies • Global/data vs. client vs. process vs. hybrid viewpoint • Focus on client viewpoint in this presentation • ACID vs. BASE • Roughly comparable to strong vs. weak consistency • ACID defined in ANSI SQL Standard • BASE introduced in [Bre2000], very weak consistency model
30. What are relevant models?
31. Non-transactional models
32. Eventual consistency • In the absence of writes, all replicas will eventually converge • Anomalies and temporal inconsistencies can happen • Must be handled at the application level • Challenging model from a software developer’s perspective • Does not need any coordination • Extremely scalable, very low latency possible • Examples: Cassandra, Amazon Dynamo, Google Big Table
33. Session consistency • Client always observes results of its past operations • No guarantees regarding global data reconciliation • Anomalies and temporal inconsistencies can happen • Must be handled at the application level • Can be challenging from a software developer’s perspective • Does not need any coordination • Very scalable, very low latency possible • Examples: Write-behind cache
34. Causal consistency • Causally related operations are seen in the same order by all clients, non-related operations can be seen in any order • Better guarantees, but a few surprises remain • Causal ordering can be different than expected • Non-related updates may be seen differently by clients • Strongest coordination-free model • Good compromise between availability and consistency • Examples: Systems based on vector clocks
35. Linearizability • Operations appear to happen in a global, real-time order • Great model, but single-value, not transactional • Temporal data inconsistencies can be observed • Best when single-value updates need to be strictly ordered • Requires strong coordination across replicas • Does not scale well, restricted availability • Examples: Paxos, Raft (etcd), ZAB (ZooKeeper)
36. Transactional models
37. Transactional consistency models Anomaly / Isolation level Dirty read Lost update Fuzzy read Phantom read Read skew Write skew Possible Possible Possible Possible Possible Possible Read committed Not possible Possible Possible Possible Possible Possible Repeatable read Not possible Not possible Not possible Possible Not possible Not possible Snapshot isolation Not possible Not possible Not possible Sometimes possible Not possible Possible Serializable Not possible Not possible Not possible Not possible Not possible Not possible Read uncommitted
38. More consistency related stuff ...
39. Quorums R + W > N W > N/2 • Used for tunable consistency • Several implementation variants available • Ranging from eventual consistency to serializability • Depends on concrete implementation • Often leads to confusion
40. CRDT • Conflict-Free Replicated Data Type • Specially designed data type • Ensures eventual consistency without coordination • Robust against lost updates • Can handle offline scenarios (e.g., mobile apps) • Not a universal tool • Not all common data types can be modeled as CRDT • Requires a lot of metadata • No widespread support • Modeling on your own can become cumbersome
41. CAP Theorem • Consistency – Availability – Partition tolerance • Defined by Eric Brewer [Bre2000] • Often misunderstood as either-or choice • Partition tolerance in not optional in distributed system • Consistency and availability define a spectrum • Only relevant when partition occurs • Behavior during partition and reconciliation are interesting • Note: Consistency defined differently from ACID definition
42. Saga Pattern • Pattern commonly misunderstood • Not a drop-in replacement for distributed transactions • Usually, 100% reliable infrastructure implicitly assumed • Correct implementation requires a lot of effort • Distributed failure modes must be considered • Persistent failure mode must be considered • Requires synchronous communication (no messaging) • Asynchronous communication requires different means • Consider Memory-Guesses-Apologies [Hel+2009] see also https://speakerdeck.com/ufried/beyond-the-saga-pattern
43. When should I use what?
44. Recommendation #1: Use a RDBMS unless infeasible The two superpowers of RDBMS are ACID transactions and the WHERE clause (plus indices & great optimizers)
45. Recommendation #2: Leverage quorums to ensure eventual consistency But make sure you handle the anomalies at application level
46. Recommendation #3: Use the consistency tools the data store provides Implementing the required consistency manually can be painful – better choose the data store based on your consistency requirements
47. Recommendation #4: Read the documentation very carefully Never trust vendor promises. Check additional sources like https://jepsen.io/analyses
48. What are common pitfalls?
49. Common pitfalls 1. Implicitly assuming strong consistency 2. Implicitly assuming isolation 3. Leaving it all to the data store 4. One-size-fits-all (the same consistency level everywhere) 5. Not testing failure scenarios 6. Trusting clocks
50. Where can I read more?
51. Further reading Consistency models – Overview [Cha2026] Kumar Chandrakant, "Introduction to Consistency Models", https://www.baeldung.com/cs/consistency-models [Col2015a] Adrian Colyer, "Out of the Fire Swamp”, Part I, http://blog.acolyer.org/2015/09/08/out-of-the-fire-swamp-part-i-the-data-crisis/ [Col2015b] Adrian Colyer, "Out of the Fire Swamp”, Part II, http://blog.acolyer.org/2015/09/09/out-of-the-fire-swamp-part-ii-peering-into-the-mist/ [Col2015c] Adrian Colyer, "Out of the Fire Swamp”, Part III, http://blog.acolyer.org/2015/09/10/out-of-the-fire-swamp-part-iii-go-with-the-flow/ [Fri2017] Uwe Friedrichsen, "Real-World Consistency Explained", https://www.slideshare.net/slideshow/realworldconsistency-explained-80984942/80984942 [Har2026] Curtis Harrison, "Data Consistency Models: From Strong Consistency to Eventual Consistency", https://veduis.com/blog/data-consistency-models/ [Jep] Jepsen, "Consistency models", https://jepsen.io/consistency/models [Ode2025] Andrew Odendaal, "Data Consistency Models in Distributed Systems", https://andrewodendaal.com/data-consistency-distributed-systems/
52. Further reading Consistency models – Deepening [Ald+2019] Hesam Aldin et al., "Consistency models in distributed systems", 2019 [Alm2024] Paulo Almeida, "A Framework for Consistency Models in Distributed Systems", INESC TE, 2024 [Bai+2013a] Peter Bailis et al., "HAT, not CAP: Towards Highly Available Transactions", HotOS 2013 [Bai+2013b] Peter Bailis et al., "Bolt-on Causal Consistency", SIGMOD 2013 [Bai+2014a] Peter Bailis et al., "Highly Available Transactions: Virtues and Limitations", VLDB 2014 [Bai+2014b] Peter Bailis et al., "Scalable Atomic Visibility with RAMP Transactions", SIGMOD 2014 [Bai+2015] Peter Bailis et al., "Coordination Avoidance in Database Systems", VLDB 2015 [Bal+2015] Valter Balegas et al., "Putting Consistency Back into Eventual Consistency", EuroSys 2015 [Ber+1995] Hal Berenson et al., "A Critique of ANSI SQL Isolation Levels", Microsoft Research, Technical Report MSR-TR-95-51, June 1995
53. Further reading Consistency models – Deepening [Cor+2012] James C. Corbett et al., "Spanner: Google’s Globally-Distributed Database", OSDI 2012 [Hel 2005] Pat Helland, “Life beyond distributed transactions”, CIDR ‘07, 2007 [Lam1979] Leslie Lamport, "How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs", IEEE Transactions on Computers, September 1979 [Lee+2015] Collin Lee et al., "Implementing Linearizability at Large Scale and Low Latency", SOSP 2015 [Sha2018] Marc Shapiro, "Database consistency models", Inria, 2018 [Vio+2016] Poalo Viotti et al., "Consistency in Non-Transactional Distributed Storage Systems", 2016 [Xie+2015] Chao Xie et al., "High-Performance ACID via Modular Concurrency Control", SOSP 2015 [Zha+2015], Irene Zhang et al., "Building Consistent Transactions with Inconsistent Replication", SOSP 2015
54. Further reading Quorums [Fri2015] Uwe Friedrichsen, "Self-Healing Data", https://www.slideshare.net/slideshow/self-healing-data/33182331 [Gif1979] David K. Gifford, "Weighted voting for replicated data", SOSP 1979 [Ske1982] Dale Skeen, "A Quorum-based Commit Protocol", Cornell University, 1982 CRDT [Fri2015] Uwe Friedrichsen, "Self-Healing Data", https://www.slideshare.net/slideshow/self-healing-data/33182331 [Lam1978] Leslie Lamport, "Time, clocks, and the ordering of events in a distributed system", Communications of the ACM, 1978 [Sha+2011a] Marc Shapiro et al., "A comprehensive study of Convergent and Commutative Replicated Data Types", Inria Research report, 2011 [Sha+2011b] Marc Shapiro et al., "Conflict-free Replicated Data Types", Inria Research report, 2011
55. Further reading CAP [Bre2000] Eric A. Brewer, "Towards Robust Distributed Systems", PODC 2000 [Bre2012] Eric A. Brewer, "CAP Twelve Years Later", IEEE Computing Now, February 2012 Saga pattern [Bai+2015] Peter Bailis et al., “Feral Concurrency Control: An Empirical Investigation of Modern Application Integrity”, SIGMOD 2015, 2015 [Fri2024] Uwe Friedrichsen, "Beyond the Saga Pattern", https://speakerdeck.com/ufried/beyond-the-saga-pattern [Gar+1987] Hector Garcia-Molina, Kenneth Salem, ”Sagas", SIGMOD ‘87, 1987 [Hel+2009] Pat Helland & Dave Campbell, “Building on Quicksand”, CIDR ‘09 Perspectives, 2009 [McC2015] Caitie McCaffrey, “Applying the Saga Pattern”, GOTO Chicago Conference 2015, https://www.youtube.com/watch?v=xDuwrtwYHu8 [McC2017] Caitie McCaffrey, ”Distributed Sagas", J On The Beach 2017, https://www.youtube.com/watch?v=0UTOLRTwOX0
56. Consistency?
57. Bring it on!
58. Uwe Friedrichsen Works @ codecentric https://bsky.app/profile/ufried.bsky.social https://www.speakerdeck.com/ufried https://ufried.com/