Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MongoDB Transaction Pattern
Search
Ian Yang
May 20, 2013
Technology
0
400
MongoDB Transaction Pattern
Ian Yang
May 20, 2013
Tweet
Share
More Decks by Ian Yang
See All by Ian Yang
Migrate to JRuby
doitian
4
770
Tmux Cheatsheet
doitian
2
310
Javascript Test Frameworks
doitian
1
140
Backbone Intro
doitian
0
230
A Tour of Go
doitian
4
900
Other Decks in Technology
See All in Technology
フルカイテン株式会社 エンジニア向け採用資料
fullkaiten
0
9.1k
許しとアジャイル
jnuank
1
140
10年の共創が示す、これからの開発者と企業の関係 ~ Crossroad
soracom
PRO
1
650
やる気のない自分との向き合い方/How to Deal with Your Unmotivated Self
sanogemaru
0
430
[Keynote] What do you need to know about DevEx in 2025
salaboy
0
130
Goに育てられ開発者向けセキュリティ事業を立ち上げた僕が今向き合う、AI × セキュリティの最前線 / Go Conference 2025
flatt_security
0
360
社内報はAIにやらせよう / Let AI handle the company newsletter
saka2jp
8
1.2k
定期的な価値提供だけじゃない、スクラムが導くチームの共創化 / 20251004 Naoki Takahashi
shift_evolve
PRO
4
350
JAZUG 15周年記念 × JAT「AI Agent開発者必見:"今"のOracle技術で拡張するAzure × OCIの共存アーキテクチャ」
shisyu_gaku
0
130
AWS Top Engineer、浮いてませんか? / As an AWS Top Engineer, Are You Out of Place?
yuj1osm
2
170
o11yで育てる、強い内製開発組織
_awache
3
130
Shirankedo NOCで見えてきたeduroam/OpenRoaming運用ノウハウと課題 - BAKUCHIKU BANBAN #2
marokiki
0
170
Featured
See All Featured
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
45
2.5k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.5k
Six Lessons from altMBA
skipperchong
28
4k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Keith and Marios Guide to Fast Websites
keithpitt
411
23k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
6.1k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
Code Reviewing Like a Champion
maltzj
525
40k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
Transcript
MongoDB Transaction Pattern Who? Ian Yang From? Intridea Inc. When?
2013-05-15 Wed
Outline 1 Overview 2 Write Operation Properties 3 Two Phrase
Commit 4 Recovering 5 Rollback 6 Multiple Applications 7 References
Transaction (ACID) Atomicity all or nothing. Consistency does not violate
any integrity constraints Isolation allow concurrent execution of transactions Durability Transaction is persistent after commit
MongoDB Features Atomicity Only on the level of a single
document. Write Concern Return write operation result
MongoDB Transaction Atomicity done or rollback Consistency rollback may leave
system in inconsistent state. Isolation other transactions can see intermediate result. Durability write concern, replica set, journal.
Requirements of Write Operations in Transaction Revertible for recover and
rollback Idempotent for resume, recover and rollback
Not Revertible db.users.save(user); db.users.update ({ username: "ian"}, user); db.users.update ({
username: "ian"}, {$set: {balance: 120}});
Revertible db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update ({
username: "ian"}, {$inc: {balance: -20}}); db.users.update ({ username: "ian"}, {$push: {tx: 20}}); db.users.update ({ username: "ian"}, {$pull: {tx: 20}});
Not Idempotent db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update
({ username: "ian"}, {$push: {tx: 20}}); db.users.update( {username: "ian", balance: 100}, {$inc: {balance: 20}} );
Idempotent db.users.update( {username: "ian", tx: {$ne: tx._id}}, {$inc: {balance: 20},
$push: {tx: tx._id}} ); db.users.update( {username: "ian", tx: tx._id}, {$inc: {balance: -20}, $pull: {tx: tx._id}} );
Sample Transfer 20 dollars from Ian to Daniel
Create Transaction Transaction provides a unique identifier, enough information to
recover and rollback, a document for atomic write operation.
var tx = { _id: ObjectId (), source: ian._id ,
dest: daniel._id , amount: 20 }; db.transactions.create(tx);
Transaction State Machine State represents transaction stages Different actions should
be taken in different stages to recover or rollback
Step 1: Create Transaction var tx = { _id: ObjectId
(), source: ian._id , dest: daniel._id , amount: 20, state: 'initial ' }; db.transactions.create(tx);
Step 2: initial → pending db.transactions.update( {_id: tx._id , state:
'initial '}, {$set: {state: 'pending '}} );
Step 3: Apply Transaction db.users.update( {_id: tx.source , tx: {$ne:
tx._id}}, {$push: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.dest , tx: {$ne: tx._id}}, {$push: {tx: tx._id}, $inc: { balance: tx.amount }} );
Step 4: pending → committed db.transactions.update( {_id: tx._id , state:
'pending '}, {$set: {state: 'committed '}} );
Step 5: Remove Pending Transaction db.users.update( {_id: tx.source , tx:
tx._id}, {$pull: {tx: tx._id}} ); db.users.update( {_id: tx.dest , tx: tx._id}, {$pull: {tx: tx._id}} );
Step 6: committed → done db.transactions.update( {_id: tx._id , state:
'committed '}, {$set: {state: 'done'}} );
Recover Initial Transactions Recover from Step 2: initial → pending
Recover Pending Transactions Recover from Step 3: Apply Transactions
Recover Committed Transactions Recover from Step 5: Remove Pending Transaction
General Rule Recover from the Step just following the step
transaction just entered current state. State transition creates a check point
Rollback Operations Better to create a new transaction in reverse
direction for Committed Transactions and Done Transactions Use following steps to rollback initial and pending transactions
Rollback 1: state → canceling db.transactions.update( {_id: t._id , state:
{$in: ['initial ', ' pending ']}}, {$set: {state: 'canceling '}} );
Rollback 2: Undo the Transaction db.users.update( {_id: tx.dest , tx:
{$ne: tx._id}}, {$pull: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.source , tx: tx._id}, {$pull: {tx: tx._id}, $inc: { balance: tx.amount }} );
Rollback 3: canceling → canceled db.transactions.update( {_id: t._id , state:
{$in: ['canceling ']}}, {$set: {state: 'canceled '}} );
Recover Rollback Restart from Rollback 2 for canceling transactions
Conclusion Create a document for any transaction Store transaction info
in the document Apply only Revertible and Idempotent operations
Conclusion (Cont.) Transit state to save current work Transit state
if several operation branches exists
Enough? Only 1 application can handle a give transaction at
any point in time.
Background A picks a pending transaction to recover B picks
also a pending transaction to recover
Scenario 1 A is so slow, when it starts apply
transaction, B already has done the transaction. A must rollback. If A exists abnormally, the transaction is in inconsistent state.
Scenario 2 B just finished Step 5: Removing Pending Transaction
A applied transactions and exited abnormally B change state to done without error
Solution findAndModify lock db.transactions.findAndModify ({ query: {state: "initial", application: {$exists:
0}}, update: {$set: {state: "pending", application: "A1"}}, new: true });
Lock Release db.transactions.findAndModify ({ query: {application: "A1"}, update: {$unset: {application:
1}}, new: true });
Lock Forever? Application exists abnormally before release the lock Solution:
timeout
Timeout Lock now = new Date(); timeout = new Date(now.valueOf
() + 60 * 10 * 1000); db.transactions.findAndModify ({ query: {state: "initial", lockUntil: {$not: {$gte: now}}}, update: {$set: {state: "pending", lockUntil: timeout}}, new: true });
Client Requirements Client timeout to ensure the operation is finished
in expected time. Check time after return from block IO Defer transactions cleanup
References http://docs.mongodb.org/ manual/tutorial/ perform-two-phase-commits/ http://docs.mongodb.org/ manual/tutorial/ isolate-sequence-of-operations/
Q & A