Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
530
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
310
Designing and implementing GraphQL API
mariuszgil
1
55
Discovering unknown with EventStorming ConFoo
mariuszgil
0
270
Game of Developer Life... Deconstructed
mariuszgil
1
160
Back to forgotten roots
mariuszgil
1
380
Go micro with microservices
mariuszgil
5
620
Machine Learning for the rescue
mariuszgil
0
380
Discovering graph structures
mariuszgil
3
530
Introduction to Aerospike with PHP
mariuszgil
8
780
Other Decks in Programming
See All in Programming
なぜ適用するか、移行して理解するClean Architecture 〜構造を超えて設計を継承する〜 / Why Apply, Migrate and Understand Clean Architecture - Inherit Design Beyond Structure
seike460
PRO
1
280
ktr0731/go-mcpでMCPサーバー作ってみた
takak2166
0
170
Select API from Kotlin Coroutine
jmatsu
1
180
Benchmark
sysong
0
230
The Evolution of Enterprise Java with Jakarta EE 11 and Beyond
ivargrimstad
1
850
KotlinConf 2025 現地で感じたServer-Side Kotlin
n_takehata
1
220
レガシーシステムの機能調査・開発におけるAI利活用
takuya_ohtonari
0
610
複数アプリケーションを育てていくための共通化戦略
irof
10
4k
C++20 射影変換
faithandbrave
0
500
What Spring Developers Should Know About Jakarta EE
ivargrimstad
0
120
生成AIコーディングとの向き合い方、AIと共創するという考え方 / How to deal with generative AI coding and the concept of co-creating with AI
seike460
PRO
1
320
エラーって何種類あるの?
kajitack
5
280
Featured
See All Featured
Mobile First: as difficult as doing things right
swwweet
223
9.7k
Designing Experiences People Love
moore
142
24k
How to train your dragon (web standard)
notwaldorf
92
6.1k
It's Worth the Effort
3n
184
28k
How STYLIGHT went responsive
nonsquared
100
5.6k
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.5k
Making Projects Easy
brettharned
116
6.3k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.8k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
107
19k
How to Ace a Technical Interview
jacobian
277
23k
Being A Developer After 40
akosma
90
590k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]