Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
540
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
310
Designing and implementing GraphQL API
mariuszgil
1
66
Discovering unknown with EventStorming ConFoo
mariuszgil
0
280
Game of Developer Life... Deconstructed
mariuszgil
1
170
Back to forgotten roots
mariuszgil
1
390
Go micro with microservices
mariuszgil
5
630
Machine Learning for the rescue
mariuszgil
0
390
Discovering graph structures
mariuszgil
3
540
Introduction to Aerospike with PHP
mariuszgil
8
800
Other Decks in Programming
See All in Programming
Server Less Code More - コードを書かない時代に生きるサーバーレスデザイン / server-less-code-more
gawa
5
1.9k
ポスターセッション: 「まっすぐ行って、右!」って言ってラズパイカーを動かしたい 〜生成AI × Raspberry Pi Pico × Gradioの試作メモ〜
komofr
0
870
CSC509 Lecture 01
javiergs
PRO
1
430
詳しくない分野でのVibe Codingで困ったことと学び/vibe-coding-in-unfamiliar-area
shibayu36
2
1.2k
Reactをクライアントで使わない
yusukebe
7
6.3k
猫と暮らすネットワークカメラ生活🐈 ~Vision frameworkでペットを愛でよう~ / iOSDC Japan 2025
yutailang0119
0
210
Introducing ReActionView: A new ActionView-Compatible ERB Engine @ Kaigi on Rails 2025, Tokyo, Japan
marcoroth
3
760
高度なUI/UXこそHotwireで作ろう Kaigi on Rails 2025
naofumi
4
2.8k
CSC305 Lecture 02
javiergs
PRO
1
260
AIを活用したレシート読み取り機能の開発から得られた実践知 / AI Receipt Scan Practice
rockname
2
1.5k
ネイティブ製ガントチャートUIを作って学ぶUICollectionViewLayoutの威力
jrsaruo
0
110
PostgreSQLで手軽にDuckDBを使う!DuckDB&pg_duckdb入門/osk2025-duckdb
takahashiikki
1
230
Featured
See All Featured
Speed Design
sergeychernyshev
32
1.1k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
Large-scale JavaScript Application Architecture
addyosmani
513
110k
How to Think Like a Performance Engineer
csswizardry
27
2k
It's Worth the Effort
3n
187
28k
Building an army of robots
kneath
306
46k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.2k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Into the Great Unknown - MozCon
thekraken
40
2.1k
Faster Mobile Websites
deanohume
310
31k
Imperfection Machines: The Place of Print at Facebook
scottboms
269
13k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]