Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling MongoDB | Sergey Gavruk
Search
Minsk MongoDB User Group
October 04, 2012
Programming
2
180
Scaling MongoDB | Sergey Gavruk
Sergey Gavruk
Meetup #7
Minsk MongoDB User Group
October 04, 2012
Tweet
Share
More Decks by Minsk MongoDB User Group
See All by Minsk MongoDB User Group
MongoDB by Chef | Yauhen Artsiukhou
bymongo
0
130
MongoDB at IronMQ | Alexander Kolesen
bymongo
0
850
Event sourcing + CQRS + MongoDB | Alex Shkor
bymongo
1
650
How it works. Indexes | Kirill Duborenko
bymongo
5
280
Aggregation Framework | Mikhail Burtylev
bymongo
1
98
MongoDB 2.2: Release update + Roadmap | Alvin Richards
bymongo
1
100
Meetup#6 Intro | Alex Litvinok
bymongo
1
47
Deploying MongoDB on Amazon WS | Michael Karpitsky
bymongo
2
110
About the problem of DBMS choice & what to do if you have gone the wrong way | Roman Bugaev
bymongo
3
120
Other Decks in Programming
See All in Programming
Namespace and Its Future
tagomoris
6
710
AIと私たちの学習の変化を考える - Claude Codeの学習モードを例に
azukiazusa1
11
4.4k
Amazon RDS 向けに提供されている MCP Server と仕組みを調べてみた/jawsug-okayama-2025-aurora-mcp
takahashiikki
1
120
ぬるぬる動かせ! Riveでアニメーション実装🐾
kno3a87
1
230
機能追加とリーダー業務の類似性
rinchoku
2
1.3k
OSS開発者という働き方
andpad
5
1.7k
RDoc meets YARD
okuramasafumi
4
170
Flutter with Dart MCP: All You Need - 박제창 2025 I/O Extended Busan
itsmedreamwalker
0
150
testingを眺める
matumoto
1
140
Design Foundational Data Engineering Observability
sucitw
3
210
rage against annotate_predecessor
junk0612
0
170
為你自己學 Python - 冷知識篇
eddie
1
350
Featured
See All Featured
Imperfection Machines: The Place of Print at Facebook
scottboms
268
13k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
820
The Straight Up "How To Draw Better" Workshop
denniskardys
236
140k
Code Reviewing Like a Champion
maltzj
525
40k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.2k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
1.6k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.5k
4 Signs Your Business is Dying
shpigford
184
22k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
30
9.7k
A Tale of Four Properties
chriscoyier
160
23k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Transcript
Scaling Sergey Gavruk @gavruk
Scaling • Ver2cal • Horizontal • By op2miza2on – Op2mize
your queries, schema, indexes – Tune you file system – Choose right disks
Share nothing architecture • Michael Stonebraker First
implementa2on in 1983 Google calls this “Sharding”
Sharding goals • App doesn’t know about clusters • Cluster
should always be available for reads and writes • Cluster should grow easily
Sharding features • Range-‐based data par22oning • Automa2c
data volume distribu2on • Transparent query rou2ng
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”) [“d”, “g”) 100 GB 500 GB 100 GB 100 GB 100 GB 400 GB 200 GB 100 GB
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “d”) 300 [“g”, “k”) 300 [“m”,
“s”) [“s”, “z”) 400 GB 400 GB 100 GB 100 GB [“d”, “g”) 100 [“k”, “m”) 100
None
Chunks -‐∞ +∞
Chunks -‐∞ +∞
null Numbers Strings Objects Arrays
binary data ObjectIds booleans Dates regular expressions smaller bigger
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2
Balancer goals • keep data distributed • minimize the amount
of data transfered
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2 Move chunk X to shard 2
Balancing Number of chunks Migra:on threshold <
20 2 21-‐80 4 80+ 8
Balancing schedule db.seangs.update({ _id : "balancer" },
{ $set : { ac2veWindow : { start : "23:00", stop : "6:00" } } }, true )
Routed Request mongos Shard 1 Shard 2
Shard 3
mongos Shard 1 Shard 2 Shard 3
Request without shard key
Without shard key + sor2ng mongos Shard 1
Shard 2 Shard 3
Consider the shard cluster if: • Data exceeds the
storage capacity of a single node • Size of working set will soon exceed your RAM • Large amount of writes
Restric2ons • You cannot update a shard key
• You must use a shard key for a single update • Index on shard key
Ideal shard key • easily divisible. • will
distribute write opera2ons among the cluster • will make it possible for the mongos to return most query opera2ons directly from a single specific mongod instance
Choosing a shard key {
_id: "1", user_id: "2345652221", date_2me: "2012-‐10-‐04“, tweet_text: “Hello world” } Reliability
Choosing a shard key Ascending { TimeStamp:
12355232, … }
Choosing a shard key Low-‐cardinality key {
Con:nent: “Europe”, Name: “Tom”, … } Zip code?
Demo
Any questions? mailto:
[email protected]