Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling MongoDB | Sergey Gavruk
Search
Minsk MongoDB User Group
October 04, 2012
Programming
2
170
Scaling MongoDB | Sergey Gavruk
Sergey Gavruk
Meetup #7
Minsk MongoDB User Group
October 04, 2012
Tweet
Share
More Decks by Minsk MongoDB User Group
See All by Minsk MongoDB User Group
MongoDB by Chef | Yauhen Artsiukhou
bymongo
0
120
MongoDB at IronMQ | Alexander Kolesen
bymongo
0
840
Event sourcing + CQRS + MongoDB | Alex Shkor
bymongo
1
640
How it works. Indexes | Kirill Duborenko
bymongo
5
270
Aggregation Framework | Mikhail Burtylev
bymongo
1
92
MongoDB 2.2: Release update + Roadmap | Alvin Richards
bymongo
1
90
Meetup#6 Intro | Alex Litvinok
bymongo
1
44
Deploying MongoDB on Amazon WS | Michael Karpitsky
bymongo
2
110
About the problem of DBMS choice & what to do if you have gone the wrong way | Roman Bugaev
bymongo
3
120
Other Decks in Programming
See All in Programming
NSOutlineView何もわからん:( 前編 / I Don't Understand About NSOutlineView :( Pt. 1
usagimaru
0
340
Amazon Bedrock Agentsを用いてアプリ開発してみた!
har1101
0
340
Why Jakarta EE Matters to Spring - and Vice Versa
ivargrimstad
0
1.2k
AI時代におけるSRE、 あるいはエンジニアの生存戦略
pyama86
6
1.2k
ActiveSupport::Notifications supporting instrumentation of Rails apps with OpenTelemetry
ymtdzzz
1
250
CSC509 Lecture 12
javiergs
PRO
0
160
Arm移行タイムアタック
qnighy
0
340
どうして僕の作ったクラスが手続き型と言われなきゃいけないんですか
akikogoto
1
120
Make Impossible States Impossibleを 意識してReactのPropsを設計しよう
ikumatadokoro
0
270
見せてあげますよ、「本物のLaravel批判」ってやつを。
77web
7
7.8k
Remix on Hono on Cloudflare Workers
yusukebe
1
300
みんなでプロポーザルを書いてみた
yuriko1211
0
280
Featured
See All Featured
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
506
140k
Scaling GitHub
holman
458
140k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
191
16k
Designing for humans not robots
tammielis
250
25k
How to train your dragon (web standard)
notwaldorf
88
5.7k
Adopting Sorbet at Scale
ufuk
73
9.1k
Building Your Own Lightsaber
phodgson
103
6.1k
KATA
mclloyd
29
14k
Docker and Python
trallard
40
3.1k
Visualization
eitanlees
145
15k
Building an army of robots
kneath
302
43k
Transcript
Scaling Sergey Gavruk @gavruk
Scaling • Ver2cal • Horizontal • By op2miza2on – Op2mize
your queries, schema, indexes – Tune you file system – Choose right disks
Share nothing architecture • Michael Stonebraker First
implementa2on in 1983 Google calls this “Sharding”
Sharding goals • App doesn’t know about clusters • Cluster
should always be available for reads and writes • Cluster should grow easily
Sharding features • Range-‐based data par22oning • Automa2c
data volume distribu2on • Transparent query rou2ng
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”) [“d”, “g”) 100 GB 500 GB 100 GB 100 GB 100 GB 400 GB 200 GB 100 GB
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “d”) 300 [“g”, “k”) 300 [“m”,
“s”) [“s”, “z”) 400 GB 400 GB 100 GB 100 GB [“d”, “g”) 100 [“k”, “m”) 100
None
Chunks -‐∞ +∞
Chunks -‐∞ +∞
null Numbers Strings Objects Arrays
binary data ObjectIds booleans Dates regular expressions smaller bigger
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2
Balancer goals • keep data distributed • minimize the amount
of data transfered
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2 Move chunk X to shard 2
Balancing Number of chunks Migra:on threshold <
20 2 21-‐80 4 80+ 8
Balancing schedule db.seangs.update({ _id : "balancer" },
{ $set : { ac2veWindow : { start : "23:00", stop : "6:00" } } }, true )
Routed Request mongos Shard 1 Shard 2
Shard 3
mongos Shard 1 Shard 2 Shard 3
Request without shard key
Without shard key + sor2ng mongos Shard 1
Shard 2 Shard 3
Consider the shard cluster if: • Data exceeds the
storage capacity of a single node • Size of working set will soon exceed your RAM • Large amount of writes
Restric2ons • You cannot update a shard key
• You must use a shard key for a single update • Index on shard key
Ideal shard key • easily divisible. • will
distribute write opera2ons among the cluster • will make it possible for the mongos to return most query opera2ons directly from a single specific mongod instance
Choosing a shard key {
_id: "1", user_id: "2345652221", date_2me: "2012-‐10-‐04“, tweet_text: “Hello world” } Reliability
Choosing a shard key Ascending { TimeStamp:
12355232, … }
Choosing a shard key Low-‐cardinality key {
Con:nent: “Europe”, Name: “Tom”, … } Zip code?
Demo
Any questions? mailto:
[email protected]