Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
540
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
310
Designing and implementing GraphQL API
mariuszgil
1
68
Discovering unknown with EventStorming ConFoo
mariuszgil
0
290
Game of Developer Life... Deconstructed
mariuszgil
1
170
Back to forgotten roots
mariuszgil
1
390
Go micro with microservices
mariuszgil
5
650
Machine Learning for the rescue
mariuszgil
0
390
Discovering graph structures
mariuszgil
3
540
Introduction to Aerospike with PHP
mariuszgil
8
810
Other Decks in Programming
See All in Programming
釣り地図SNSにおける有料機能の実装
nokonoko1203
0
200
Range on Rails ―「多重範囲型」という新たな選択肢が、複雑ロジックを劇的にシンプルにしたワケ
rizap_tech
0
7.7k
あなたとKaigi on Rails / Kaigi on Rails + You
shimoju
0
190
TFLintカスタムプラグインで始める Terraformコード品質管理
bells17
2
440
kiroとCodexで最高のSpec駆動開発を!!数時間で web3ネイティブなミニゲームを作ってみたよ!
mashharuki
0
910
Go言語の特性を活かした公式MCP SDKの設計
hond0413
2
520
なぜGoのジェネリクスはこの形なのか? - Featherweight Goが明かす設計の核心
qualiarts
0
250
Vueのバリデーション、結局どれを選べばいい? ― 自作バリデーションの限界と、脱却までの道のり ― / Which Vue Validation Library Should We Really Use? The Limits of Self-Made Validation and How I Finally Moved On
neginasu
2
1.6k
Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job
hollycummins
0
250
品質ワークショップをやってみた
nealle
0
640
Claude CodeによるAI駆動開発の実践 〜そこから見えてきたこれからのプログラミング〜
iriikeita
0
340
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
270
Featured
See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
80
6k
Building an army of robots
kneath
305
46k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
130k
The Art of Programming - Codeland 2020
erikaheidi
56
14k
GitHub's CSS Performance
jonrohan
1032
470k
What's in a price? How to price your products and services
michaelherold
246
12k
Learning to Love Humans: Emotional Interface Design
aarron
274
41k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.6k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
230
22k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]