Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling MongoDB | Sergey Gavruk
Search
Minsk MongoDB User Group
October 04, 2012
Programming
2
180
Scaling MongoDB | Sergey Gavruk
Sergey Gavruk
Meetup #7
Minsk MongoDB User Group
October 04, 2012
Tweet
Share
More Decks by Minsk MongoDB User Group
See All by Minsk MongoDB User Group
MongoDB by Chef | Yauhen Artsiukhou
bymongo
0
130
MongoDB at IronMQ | Alexander Kolesen
bymongo
0
850
Event sourcing + CQRS + MongoDB | Alex Shkor
bymongo
1
640
How it works. Indexes | Kirill Duborenko
bymongo
5
280
Aggregation Framework | Mikhail Burtylev
bymongo
1
97
MongoDB 2.2: Release update + Roadmap | Alvin Richards
bymongo
1
99
Meetup#6 Intro | Alex Litvinok
bymongo
1
46
Deploying MongoDB on Amazon WS | Michael Karpitsky
bymongo
2
110
About the problem of DBMS choice & what to do if you have gone the wrong way | Roman Bugaev
bymongo
3
120
Other Decks in Programming
See All in Programming
Bytecode Manipulation 으로 생산성 높이기
bigstark
2
380
都市をデータで見るってこういうこと PLATEAU属性情報入門
nokonoko1203
1
570
KotlinConf 2025 現地で感じたServer-Side Kotlin
n_takehata
1
230
Benchmark
sysong
0
270
Blazing Fast UI Development with Compose Hot Reload (droidcon New York 2025)
zsmb
1
220
CursorはMCPを使った方が良いぞ
taigakono
1
180
設計やレビューに悩んでいるPHPerに贈る、クリーンなオブジェクト設計の指針たち
panda_program
6
1.4k
AIコーディング道場勉強会#2 君(エンジニア)たちはどう生きるか
misakiotb
1
250
Kotlin エンジニアへ送る:Swift 案件に参加させられる日に備えて~似てるけど色々違う Swift の仕様 / from Kotlin to Swift
lovee
1
260
XP, Testing and ninja testing
m_seki
3
200
ReadMoreTextView
fornewid
1
480
Hypervel - A Coroutine Framework for Laravel Artisans
albertcht
1
100
Featured
See All Featured
Building an army of robots
kneath
306
45k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Become a Pro
speakerdeck
PRO
28
5.4k
Producing Creativity
orderedlist
PRO
346
40k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Music & Morning Musume
bryan
46
6.6k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
490
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
700
Code Review Best Practice
trishagee
68
18k
Transcript
Scaling Sergey Gavruk @gavruk
Scaling • Ver2cal • Horizontal • By op2miza2on – Op2mize
your queries, schema, indexes – Tune you file system – Choose right disks
Share nothing architecture • Michael Stonebraker First
implementa2on in 1983 Google calls this “Sharding”
Sharding goals • App doesn’t know about clusters • Cluster
should always be available for reads and writes • Cluster should grow easily
Sharding features • Range-‐based data par22oning • Automa2c
data volume distribu2on • Transparent query rou2ng
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”) [“d”, “g”) 100 GB 500 GB 100 GB 100 GB 100 GB 400 GB 200 GB 100 GB
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “d”) 300 [“g”, “k”) 300 [“m”,
“s”) [“s”, “z”) 400 GB 400 GB 100 GB 100 GB [“d”, “g”) 100 [“k”, “m”) 100
None
Chunks -‐∞ +∞
Chunks -‐∞ +∞
null Numbers Strings Objects Arrays
binary data ObjectIds booleans Dates regular expressions smaller bigger
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2
Balancer goals • keep data distributed • minimize the amount
of data transfered
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2 Move chunk X to shard 2
Balancing Number of chunks Migra:on threshold <
20 2 21-‐80 4 80+ 8
Balancing schedule db.seangs.update({ _id : "balancer" },
{ $set : { ac2veWindow : { start : "23:00", stop : "6:00" } } }, true )
Routed Request mongos Shard 1 Shard 2
Shard 3
mongos Shard 1 Shard 2 Shard 3
Request without shard key
Without shard key + sor2ng mongos Shard 1
Shard 2 Shard 3
Consider the shard cluster if: • Data exceeds the
storage capacity of a single node • Size of working set will soon exceed your RAM • Large amount of writes
Restric2ons • You cannot update a shard key
• You must use a shard key for a single update • Index on shard key
Ideal shard key • easily divisible. • will
distribute write opera2ons among the cluster • will make it possible for the mongos to return most query opera2ons directly from a single specific mongod instance
Choosing a shard key {
_id: "1", user_id: "2345652221", date_2me: "2012-‐10-‐04“, tweet_text: “Hello world” } Reliability
Choosing a shard key Ascending { TimeStamp:
12355232, … }
Choosing a shard key Low-‐cardinality key {
Con:nent: “Europe”, Name: “Tom”, … } Zip code?
Demo
Any questions? mailto:
[email protected]