Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
520
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
280
Designing and implementing GraphQL API
mariuszgil
1
41
Discovering unknown with EventStorming ConFoo
mariuszgil
0
260
Game of Developer Life... Deconstructed
mariuszgil
1
150
Back to forgotten roots
mariuszgil
1
370
Go micro with microservices
mariuszgil
5
560
Machine Learning for the rescue
mariuszgil
0
350
Discovering graph structures
mariuszgil
3
530
Introduction to Aerospike with PHP
mariuszgil
8
750
Other Decks in Programming
See All in Programming
“あなた” の開発を支援する AI エージェント Bedrock Engineer / introducing-bedrock-engineer
gawa
4
320
ドメインイベント増えすぎ問題
h0r15h0
2
570
毎日13時間もかかるバッチ処理をたった3日で60%短縮するためにやったこと
sho_ssk_
1
550
Rubyでつくるパケットキャプチャツール
ydah
0
170
ESLintプラグインを使用してCDKのセオリーを適用する
yamanashi_ren01
2
240
20241217 競争力強化とビジネス価値創出への挑戦:モノタロウのシステムモダナイズ、開発組織の進化と今後の展望
monotaro
PRO
0
290
PHPカンファレンス 2024|共創を加速するための若手の技術挑戦
weddingpark
0
140
ISUCON14感想戦で85万点まで頑張ってみた
ponyo877
1
600
月刊 競技プログラミングをお仕事に役立てるには
terryu16
1
1.2k
Flatt Security XSS Challenge 解答・解説
flatt_security
0
740
ChatGPT とつくる PHP で OS 実装
memory1994
PRO
3
190
「とりあえず動く」コードはよい、「読みやすい」コードはもっとよい / Code that 'just works' is good, but code that is 'readable' is even better.
mkmk884
6
1.4k
Featured
See All Featured
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
3
180
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
132
33k
The Language of Interfaces
destraynor
155
24k
GraphQLの誤解/rethinking-graphql
sonatard
68
10k
A better future with KSS
kneath
238
17k
Designing on Purpose - Digital PM Summit 2013
jponch
116
7.1k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
6
500
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2k
Bootstrapping a Software Product
garrettdimon
PRO
305
110k
Unsuck your backbone
ammeep
669
57k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
160
15k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]