Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling MongoDB | Sergey Gavruk
Search
Minsk MongoDB User Group
October 04, 2012
Programming
2
180
Scaling MongoDB | Sergey Gavruk
Sergey Gavruk
Meetup #7
Minsk MongoDB User Group
October 04, 2012
Tweet
Share
More Decks by Minsk MongoDB User Group
See All by Minsk MongoDB User Group
MongoDB by Chef | Yauhen Artsiukhou
bymongo
0
130
MongoDB at IronMQ | Alexander Kolesen
bymongo
0
850
Event sourcing + CQRS + MongoDB | Alex Shkor
bymongo
1
640
How it works. Indexes | Kirill Duborenko
bymongo
5
280
Aggregation Framework | Mikhail Burtylev
bymongo
1
98
MongoDB 2.2: Release update + Roadmap | Alvin Richards
bymongo
1
100
Meetup#6 Intro | Alex Litvinok
bymongo
1
46
Deploying MongoDB on Amazon WS | Michael Karpitsky
bymongo
2
110
About the problem of DBMS choice & what to do if you have gone the wrong way | Roman Bugaev
bymongo
3
120
Other Decks in Programming
See All in Programming
エンジニアのための”最低限いい感じ”デザイン入門
shunshobon
0
130
Flutterと Vibe Coding で個人開発!
hyshu
1
270
SOCI Index Manifest v2が出たので調べてみた / Introduction to SOCI Index Manifest v2
tkikuc
1
110
ライブ配信サービスの インフラのジレンマ -マルチクラウドに至ったワケ-
mirrativ
2
260
MCPで実現するAIエージェント駆動のNext.jsアプリデバッグ手法
nyatinte
7
880
Jakarta EE Core Profile and Helidon - Speed, Simplicity, and AI Integration
ivargrimstad
0
200
実践!App Intents対応
yuukiw00w
1
350
Rancher と Terraform
fufuhu
0
110
Langfuseと歩む生成AI活用推進
licux
3
300
CSC305 Summer Lecture 05
javiergs
PRO
0
110
AWS Serverless Application Model入門_20250708
smatsuzaki
0
130
TanStack DB ~状態管理の新しい考え方~
bmthd
2
340
Featured
See All Featured
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
780
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Why Our Code Smells
bkeepers
PRO
338
57k
How STYLIGHT went responsive
nonsquared
100
5.7k
Practical Orchestrator
shlominoach
190
11k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.8k
Docker and Python
trallard
45
3.5k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Documentation Writing (for coders)
carmenintech
73
5k
Automating Front-end Workflow
addyosmani
1370
200k
Being A Developer After 40
akosma
90
590k
Transcript
Scaling Sergey Gavruk @gavruk
Scaling • Ver2cal • Horizontal • By op2miza2on – Op2mize
your queries, schema, indexes – Tune you file system – Choose right disks
Share nothing architecture • Michael Stonebraker First
implementa2on in 1983 Google calls this “Sharding”
Sharding goals • App doesn’t know about clusters • Cluster
should always be available for reads and writes • Cluster should grow easily
Sharding features • Range-‐based data par22oning • Automa2c
data volume distribu2on • Transparent query rou2ng
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”) [“d”, “g”) 100 GB 500 GB 100 GB 100 GB 100 GB 400 GB 200 GB 100 GB
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “d”) 300 [“g”, “k”) 300 [“m”,
“s”) [“s”, “z”) 400 GB 400 GB 100 GB 100 GB [“d”, “g”) 100 [“k”, “m”) 100
None
Chunks -‐∞ +∞
Chunks -‐∞ +∞
null Numbers Strings Objects Arrays
binary data ObjectIds booleans Dates regular expressions smaller bigger
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2
Balancer goals • keep data distributed • minimize the amount
of data transfered
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2 Move chunk X to shard 2
Balancing Number of chunks Migra:on threshold <
20 2 21-‐80 4 80+ 8
Balancing schedule db.seangs.update({ _id : "balancer" },
{ $set : { ac2veWindow : { start : "23:00", stop : "6:00" } } }, true )
Routed Request mongos Shard 1 Shard 2
Shard 3
mongos Shard 1 Shard 2 Shard 3
Request without shard key
Without shard key + sor2ng mongos Shard 1
Shard 2 Shard 3
Consider the shard cluster if: • Data exceeds the
storage capacity of a single node • Size of working set will soon exceed your RAM • Large amount of writes
Restric2ons • You cannot update a shard key
• You must use a shard key for a single update • Index on shard key
Ideal shard key • easily divisible. • will
distribute write opera2ons among the cluster • will make it possible for the mongos to return most query opera2ons directly from a single specific mongod instance
Choosing a shard key {
_id: "1", user_id: "2345652221", date_2me: "2012-‐10-‐04“, tweet_text: “Hello world” } Reliability
Choosing a shard key Ascending { TimeStamp:
12355232, … }
Choosing a shard key Low-‐cardinality key {
Con:nent: “Europe”, Name: “Tom”, … } Zip code?
Demo
Any questions? mailto:
[email protected]