Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
530
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
300
Designing and implementing GraphQL API
mariuszgil
1
52
Discovering unknown with EventStorming ConFoo
mariuszgil
0
270
Game of Developer Life... Deconstructed
mariuszgil
1
160
Back to forgotten roots
mariuszgil
1
380
Go micro with microservices
mariuszgil
5
610
Machine Learning for the rescue
mariuszgil
0
370
Discovering graph structures
mariuszgil
3
530
Introduction to Aerospike with PHP
mariuszgil
8
780
Other Decks in Programming
See All in Programming
List Unfolding - 'unfold' as the Computational Dual of 'fold', and how 'unfold' relates to 'iterate'"
philipschwarz
PRO
0
130
ソフトウェア品質特性、意識してますか?AIの真の力を引き出す活用事例 / ai-and-software-quality
minodriven
19
6.7k
テスト分析入門/Test Analysis Tutorial
goyoki
11
2.7k
AIにコードを生成するコードを作らせて、再現性を担保しよう! / Let AI generate code to ensure reproducibility
yamachu
7
6k
型安全なDrag and Dropの設計を考える
yudppp
5
660
人には人それぞれのサービス層がある
shimabox
3
460
TSConfig Solution Style & subpath imports to switch types on a per-file basis
maminami373
1
180
DevDay2025-OracleDatabase-kernel-addressing-history
oracle4engineer
PRO
7
1.6k
AI時代のリアーキテクチャ戦略 / Re-architecture Strategy in the AI Era
dachi023
0
190
「兵法」から見る質とスピード
ickx
0
200
Proxmoxをまとめて管理できるコンソール作ってみました
karugamo
1
410
Practical Domain-Driven Design - Workshop at NDC 2025
mufrid
0
130
Featured
See All Featured
It's Worth the Effort
3n
184
28k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
A designer walks into a library…
pauljervisheath
205
24k
Fontdeck: Realign not Redesign
paulrobertlloyd
84
5.5k
Art, The Web, and Tiny UX
lynnandtonic
298
21k
RailsConf 2023
tenderlove
30
1.1k
Designing for Performance
lara
608
69k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.8k
Testing 201, or: Great Expectations
jmmastey
42
7.5k
KATA
mclloyd
29
14k
Agile that works and the tools we love
rasmusluckow
329
21k
Adopting Sorbet at Scale
ufuk
76
9.4k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]