Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MongoDB Transaction Pattern
Search
Ian Yang
May 20, 2013
Technology
0
400
MongoDB Transaction Pattern
Ian Yang
May 20, 2013
Tweet
Share
More Decks by Ian Yang
See All by Ian Yang
Migrate to JRuby
doitian
4
770
Tmux Cheatsheet
doitian
2
310
Javascript Test Frameworks
doitian
1
140
Backbone Intro
doitian
0
230
A Tour of Go
doitian
4
900
Other Decks in Technology
See All in Technology
VCC 2025 Write-up
bata_24
0
180
SwiftUIのGeometryReaderとScrollViewを基礎から応用まで学び直す:設計と活用事例
fumiyasac0921
0
140
10年の共創が示す、これからの開発者と企業の関係 ~ Crossroad
soracom
PRO
1
190
いま注目しているデータエンジニアリングの論点
ikkimiyazaki
0
590
実装で解き明かす並行処理の歴史
zozotech
PRO
1
350
動画データのポテンシャルを引き出す! Databricks と AI活用への奮闘記(現在進行形)
databricksjapan
0
150
GC25 Recap+: Advancing Go Garbage Collection with Green Tea
logica0419
1
410
From Prompt to Product @ How to Web 2025, Bucharest, Romania
janwerner
0
120
バイブコーディングと継続的デプロイメント
nwiizo
2
430
コンテキストエンジニアリングとは? 考え方と応用方法
findy_eventslides
4
900
Trust as Infrastructure
bcantrill
0
340
about #74462 go/token#FileSet
tomtwinkle
1
340
Featured
See All Featured
Building Applications with DynamoDB
mza
96
6.6k
Building an army of robots
kneath
306
46k
RailsConf 2023
tenderlove
30
1.2k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
140
34k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
Build your cross-platform service in a week with App Engine
jlugia
232
18k
How to Think Like a Performance Engineer
csswizardry
27
2k
A better future with KSS
kneath
239
17k
[RailsConf 2023] Rails as a piece of cake
palkan
57
5.9k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.7k
We Have a Design System, Now What?
morganepeng
53
7.8k
jQuery: Nuts, Bolts and Bling
dougneiner
64
7.9k
Transcript
MongoDB Transaction Pattern Who? Ian Yang From? Intridea Inc. When?
2013-05-15 Wed
Outline 1 Overview 2 Write Operation Properties 3 Two Phrase
Commit 4 Recovering 5 Rollback 6 Multiple Applications 7 References
Transaction (ACID) Atomicity all or nothing. Consistency does not violate
any integrity constraints Isolation allow concurrent execution of transactions Durability Transaction is persistent after commit
MongoDB Features Atomicity Only on the level of a single
document. Write Concern Return write operation result
MongoDB Transaction Atomicity done or rollback Consistency rollback may leave
system in inconsistent state. Isolation other transactions can see intermediate result. Durability write concern, replica set, journal.
Requirements of Write Operations in Transaction Revertible for recover and
rollback Idempotent for resume, recover and rollback
Not Revertible db.users.save(user); db.users.update ({ username: "ian"}, user); db.users.update ({
username: "ian"}, {$set: {balance: 120}});
Revertible db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update ({
username: "ian"}, {$inc: {balance: -20}}); db.users.update ({ username: "ian"}, {$push: {tx: 20}}); db.users.update ({ username: "ian"}, {$pull: {tx: 20}});
Not Idempotent db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update
({ username: "ian"}, {$push: {tx: 20}}); db.users.update( {username: "ian", balance: 100}, {$inc: {balance: 20}} );
Idempotent db.users.update( {username: "ian", tx: {$ne: tx._id}}, {$inc: {balance: 20},
$push: {tx: tx._id}} ); db.users.update( {username: "ian", tx: tx._id}, {$inc: {balance: -20}, $pull: {tx: tx._id}} );
Sample Transfer 20 dollars from Ian to Daniel
Create Transaction Transaction provides a unique identifier, enough information to
recover and rollback, a document for atomic write operation.
var tx = { _id: ObjectId (), source: ian._id ,
dest: daniel._id , amount: 20 }; db.transactions.create(tx);
Transaction State Machine State represents transaction stages Different actions should
be taken in different stages to recover or rollback
Step 1: Create Transaction var tx = { _id: ObjectId
(), source: ian._id , dest: daniel._id , amount: 20, state: 'initial ' }; db.transactions.create(tx);
Step 2: initial → pending db.transactions.update( {_id: tx._id , state:
'initial '}, {$set: {state: 'pending '}} );
Step 3: Apply Transaction db.users.update( {_id: tx.source , tx: {$ne:
tx._id}}, {$push: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.dest , tx: {$ne: tx._id}}, {$push: {tx: tx._id}, $inc: { balance: tx.amount }} );
Step 4: pending → committed db.transactions.update( {_id: tx._id , state:
'pending '}, {$set: {state: 'committed '}} );
Step 5: Remove Pending Transaction db.users.update( {_id: tx.source , tx:
tx._id}, {$pull: {tx: tx._id}} ); db.users.update( {_id: tx.dest , tx: tx._id}, {$pull: {tx: tx._id}} );
Step 6: committed → done db.transactions.update( {_id: tx._id , state:
'committed '}, {$set: {state: 'done'}} );
Recover Initial Transactions Recover from Step 2: initial → pending
Recover Pending Transactions Recover from Step 3: Apply Transactions
Recover Committed Transactions Recover from Step 5: Remove Pending Transaction
General Rule Recover from the Step just following the step
transaction just entered current state. State transition creates a check point
Rollback Operations Better to create a new transaction in reverse
direction for Committed Transactions and Done Transactions Use following steps to rollback initial and pending transactions
Rollback 1: state → canceling db.transactions.update( {_id: t._id , state:
{$in: ['initial ', ' pending ']}}, {$set: {state: 'canceling '}} );
Rollback 2: Undo the Transaction db.users.update( {_id: tx.dest , tx:
{$ne: tx._id}}, {$pull: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.source , tx: tx._id}, {$pull: {tx: tx._id}, $inc: { balance: tx.amount }} );
Rollback 3: canceling → canceled db.transactions.update( {_id: t._id , state:
{$in: ['canceling ']}}, {$set: {state: 'canceled '}} );
Recover Rollback Restart from Rollback 2 for canceling transactions
Conclusion Create a document for any transaction Store transaction info
in the document Apply only Revertible and Idempotent operations
Conclusion (Cont.) Transit state to save current work Transit state
if several operation branches exists
Enough? Only 1 application can handle a give transaction at
any point in time.
Background A picks a pending transaction to recover B picks
also a pending transaction to recover
Scenario 1 A is so slow, when it starts apply
transaction, B already has done the transaction. A must rollback. If A exists abnormally, the transaction is in inconsistent state.
Scenario 2 B just finished Step 5: Removing Pending Transaction
A applied transactions and exited abnormally B change state to done without error
Solution findAndModify lock db.transactions.findAndModify ({ query: {state: "initial", application: {$exists:
0}}, update: {$set: {state: "pending", application: "A1"}}, new: true });
Lock Release db.transactions.findAndModify ({ query: {application: "A1"}, update: {$unset: {application:
1}}, new: true });
Lock Forever? Application exists abnormally before release the lock Solution:
timeout
Timeout Lock now = new Date(); timeout = new Date(now.valueOf
() + 60 * 10 * 1000); db.transactions.findAndModify ({ query: {state: "initial", lockUntil: {$not: {$gte: now}}}, update: {$set: {state: "pending", lockUntil: timeout}}, new: true });
Client Requirements Client timeout to ensure the operation is finished
in expected time. Check time after return from block IO Defer transactions cleanup
References http://docs.mongodb.org/ manual/tutorial/ perform-two-phase-commits/ http://docs.mongodb.org/ manual/tutorial/ isolate-sequence-of-operations/
Q & A