Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Why You Shouldn't Write a Database
Search
benbjohnson
October 22, 2015
Technology
2
530
Why You Shouldn't Write a Database
Talk given at the October 2015 Denver Go meetup.
benbjohnson
October 22, 2015
Tweet
Share
More Decks by benbjohnson
See All by benbjohnson
Static Code Analysis Using SSA
benbjohnson
6
1.6k
Writing a High Performance Database in Go
benbjohnson
11
6.6k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
106
28k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
2k
New Tech - Sky
benbjohnson
2
370
Behavioral Databases
benbjohnson
16
2.4k
Sky Analytics Pitch Deck
benbjohnson
0
570
Other Decks in Technology
See All in Technology
ガチ勢によるPipeCD運用大全〜滑らかなCI/CDを添えて〜 / ai-pipecd-encyclopedia
cyberagentdevelopers
PRO
3
210
[JAWS-UG金沢支部×コンテナ支部合同企画]コンテナとは何か
furuton
3
260
WINTICKETアプリで実現した高可用性と高速リリースを支えるエコシステム / winticket-eco-system
cyberagentdevelopers
PRO
1
190
【技術書典17】OpenFOAM(自宅で極める流体解析)2次元円柱まわりの流れ
kamakiri1225
0
220
プロダクト成長に対応するプラットフォーム戦略:Authleteによる共通認証基盤の移行事例 / Building an authentication platform using Authlete and AWS
kakehashi
1
150
AWS re:Inventを徹底的に楽しむためのTips / Tips for thoroughly enjoying AWS re:Invent
yuj1osm
1
570
とあるユーザー企業におけるリスクベースで考えるセキュリティ業務のお話し
4su_para
3
330
visionOSでの空間表現実装とImmersive Video表示について / ai-immersive-visionos
cyberagentdevelopers
PRO
1
110
生成AIと知識グラフの相互利用に基づく文書解析
koujikozaki
1
140
Gradle: The Build System That Loves To Hate You
aurimas
2
150
20241031_AWS_生成AIハッカソン_GenMuck
tsumita
0
110
生成AIとAWS CDKで実現! 自社ブログレビューの効率化
ymae
2
330
Featured
See All Featured
Building Adaptive Systems
keathley
38
2.2k
A better future with KSS
kneath
238
17k
Large-scale JavaScript Application Architecture
addyosmani
510
110k
Fontdeck: Realign not Redesign
paulrobertlloyd
81
5.2k
GitHub's CSS Performance
jonrohan
1030
460k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
31
2.7k
Automating Front-end Workflow
addyosmani
1365
200k
Why Our Code Smells
bkeepers
PRO
334
57k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
355
29k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
42
9.2k
Designing for Performance
lara
604
68k
Fireside Chat
paigeccino
32
3k
Transcript
Why You Shouldn’t Write a Database Ben Johnson
Let’s define “database”
low-level LevelDB, BerkeleyDB, LMDB
Direct interface to OS Files, pages, & blocks Responsible for
data integrity Typically key/value
high-level SQLite, MySQL, PostgreSQL, Mongo, InfluxDB, etc
Builds on low-level stores Rows, tables, indexes Interfaces with end
user Relational, document, time series
Me
SkyDB ReportifyDB BoltDB InfluxDB
There are n+1 reasons not to write a database
#1. High barrier to entry
How to write a database
None
legit
sorta, but not quite
seriously?
suggests using XML
nope
Available resources?
Research papers!
Research papers! Narrowly focused (indexing, storage, locks)
Research papers! High level (little or no code)
Research papers! Assumes a Ph.D
Read source code!
Read source code! MySQL >1M SLOC
Read source code! Even small databases are 10KLOC+
There is no Writing Databases 101
#2. Debugging Sucks
Bugs are catastrophic Cause corruption, loss of data integrity
Bugs are catastrophic Users with data loss are very unhappy
Find a good hex editor
Find a good hex editor (I use Hex Fiend)
None
This is not what a database looks like
None
THIS is what a database looks like
Debugging w/o data Users usually can’t release their data
#3. Tradeoffs
Users don’t understand tradeoffs
None
Read Optimized vs Write Optimized
CPU Bound vs IO Bound
https://gist.github.com/jboner/2841832
Features are liabilities It’s not if you have bugs, it’s
how many
#4. Limited Community
Very few people who have written a production database
TONS of people who will tell you why your database
sucks
Databases are hard
The End
The End (Just kidding!)
Not scared off yet?
Only 2 reasons to write a database:
#1. To learn
Data Integrity Indexing Isolation levels Recovery Prefetching Parser & Lexers
Replication Snapshotting Transactions Materialized View Referential Integrity Query Compilation Query Planning Query Optimization Serializability Write Ahead Log Memory Profiling Redo/Undo Log Snapshot Isolation Recovery Two-Phase Commit Quorums
#2. Gain efficiency (For a specific use case)
The more generic your database, the fewer assumptions you can
make
Go write a database!
The End @benbjohnson
Questions? @benbjohnson