Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Why You Shouldn't Write a Database
Search
benbjohnson
October 22, 2015
Technology
2
540
Why You Shouldn't Write a Database
Talk given at the October 2015 Denver Go meetup.
benbjohnson
October 22, 2015
Tweet
Share
More Decks by benbjohnson
See All by benbjohnson
Static Code Analysis Using SSA
benbjohnson
6
1.7k
Writing a High Performance Database in Go
benbjohnson
11
6.7k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
106
28k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
2k
New Tech - Sky
benbjohnson
2
370
Behavioral Databases
benbjohnson
16
2.5k
Sky Analytics Pitch Deck
benbjohnson
0
590
Other Decks in Technology
See All in Technology
製造業の会計システムをDDDで開発した話
caddi_eng
3
930
ルートユーザーの活用と管理を徹底的に深掘る
yuobayashi
6
710
ソフトウェアプロジェクトの成功率が上がらない原因-「社会価値を考える」ということ-
ytanaka5569
0
120
Javaの新しめの機能を知ったかぶれるようになる話 #kanjava
irof
3
4.9k
SaaSプロダクト開発におけるバグの早期検出のためのAcceptance testの取り組み
kworkdev
PRO
0
400
新卒エンジニア研修の試行錯誤と工夫/nikkei-tech-talk-31
nishiuma
0
190
30代エンジニアが考える、エンジニア生存戦略~~セキュリティを添えて~~
masakiokuda
4
2k
OPENLOGI Company Profile for engineer
hr01
1
22k
数百台のオンプレミスのサーバーをEKSに移行した話
yukiteraoka
0
640
バクラクでのSystem Risk Records導入による変化と改善の取り組み/Changes and Improvement Initiatives Resulting from the Implementation of System Risk Records
taddy_919
0
220
技術好きなエンジニアが _リーダーへの進化_ によって得たものと失ったもの / The Gains and Losses of a Tech-Enthusiast Engineer’s “Evolution into Leadership”
kaminashi
0
200
移行できそうでやりきれなかった 10年超えのシステムを葬るための戦略
ryu955
2
200
Featured
See All Featured
The Invisible Side of Design
smashingmag
299
50k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
12
1.4k
A Philosophy of Restraint
colly
203
16k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
160
15k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Gamification - CAS2011
davidbonilla
81
5.2k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
331
21k
Faster Mobile Websites
deanohume
306
31k
Java REST API Framework Comparison - PWX 2021
mraible
29
8.5k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Six Lessons from altMBA
skipperchong
27
3.7k
Transcript
Why You Shouldn’t Write a Database Ben Johnson
Let’s define “database”
low-level LevelDB, BerkeleyDB, LMDB
Direct interface to OS Files, pages, & blocks Responsible for
data integrity Typically key/value
high-level SQLite, MySQL, PostgreSQL, Mongo, InfluxDB, etc
Builds on low-level stores Rows, tables, indexes Interfaces with end
user Relational, document, time series
Me
SkyDB ReportifyDB BoltDB InfluxDB
There are n+1 reasons not to write a database
#1. High barrier to entry
How to write a database
None
legit
sorta, but not quite
seriously?
suggests using XML
nope
Available resources?
Research papers!
Research papers! Narrowly focused (indexing, storage, locks)
Research papers! High level (little or no code)
Research papers! Assumes a Ph.D
Read source code!
Read source code! MySQL >1M SLOC
Read source code! Even small databases are 10KLOC+
There is no Writing Databases 101
#2. Debugging Sucks
Bugs are catastrophic Cause corruption, loss of data integrity
Bugs are catastrophic Users with data loss are very unhappy
Find a good hex editor
Find a good hex editor (I use Hex Fiend)
None
This is not what a database looks like
None
THIS is what a database looks like
Debugging w/o data Users usually can’t release their data
#3. Tradeoffs
Users don’t understand tradeoffs
None
Read Optimized vs Write Optimized
CPU Bound vs IO Bound
https://gist.github.com/jboner/2841832
Features are liabilities It’s not if you have bugs, it’s
how many
#4. Limited Community
Very few people who have written a production database
TONS of people who will tell you why your database
sucks
Databases are hard
The End
The End (Just kidding!)
Not scared off yet?
Only 2 reasons to write a database:
#1. To learn
Data Integrity Indexing Isolation levels Recovery Prefetching Parser & Lexers
Replication Snapshotting Transactions Materialized View Referential Integrity Query Compilation Query Planning Query Optimization Serializability Write Ahead Log Memory Profiling Redo/Undo Log Snapshot Isolation Recovery Two-Phase Commit Quorums
#2. Gain efficiency (For a specific use case)
The more generic your database, the fewer assumptions you can
make
Go write a database!
The End @benbjohnson
Questions? @benbjohnson