Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Behavioral Databases
Search
benbjohnson
October 24, 2012
Technology
16
2.5k
Behavioral Databases
A presentation I gave at the Derailed meetup in Denver on October 24th, 2012.
benbjohnson
October 24, 2012
Tweet
Share
More Decks by benbjohnson
See All by benbjohnson
Why You Shouldn't Write a Database
benbjohnson
2
550
Static Code Analysis Using SSA
benbjohnson
6
1.7k
Writing a High Performance Database in Go
benbjohnson
11
6.8k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
106
28k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
2k
New Tech - Sky
benbjohnson
2
380
Sky Analytics Pitch Deck
benbjohnson
0
600
Other Decks in Technology
See All in Technology
_第3回__AIxIoTビジネス共創ラボ紹介資料_20250617.pdf
iotcomjpadmin
0
140
Workflows から Agents へ ~ 生成 AI アプリの成長過程とアプローチ~
belongadmin
3
170
IAMのマニアックな話 2025を執筆して、 見えてきたAWSアカウント管理の現在
nrinetcom
PRO
4
660
Welcome to the LLM Club
koic
0
130
知識を整理して未来を作る 〜SKDとAI協業への助走〜
yosh1995
0
140
Navigation3でViewModelにデータを渡す方法
mikanichinose
0
210
CI/CDとタスク共有で加速するVibe Coding
tnbe21
0
230
AIエージェントの継続的改善のためオブザーバビリティ
pharma_x_tech
6
1.4k
Definition of Done
kawaguti
PRO
6
460
Prox Industries株式会社 会社紹介資料
proxindustries
0
200
20250623 Findy Lunch LT Brown
3150
0
770
Amazon S3標準/ S3 Tables/S3 Express One Zoneを使ったログ分析
shigeruoda
2
390
Featured
See All Featured
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
For a Future-Friendly Web
brad_frost
179
9.8k
Gamification - CAS2011
davidbonilla
81
5.3k
Code Reviewing Like a Champion
maltzj
524
40k
Optimizing for Happiness
mojombo
379
70k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.8k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
How to Ace a Technical Interview
jacobian
277
23k
VelocityConf: Rendering Performance Case Studies
addyosmani
330
24k
We Have a Design System, Now What?
morganepeng
52
7.6k
A Tale of Four Properties
chriscoyier
159
23k
How STYLIGHT went responsive
nonsquared
100
5.6k
Transcript
Behavioral Databases Next Generation NoSQL Analytics By Ben Johnson
My Background
Former Oracle DBA
Former Oracle DBA Behavioral Analytics
Former Oracle DBA Data Visualization Behavioral Analytics
Why did I write a database?
Other Databases Are Too Slow Redis 100K ops/second PostgreSQL 5K
QPS/core
Processing Needed to Occur Near Data Client Server 1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms 1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms 1ms 1ms 1ms 1ms
Processing Needed to Occur Near Data
Memory is getting cheap. Use it. 2010 $20/GB http://www.jcmit.com/memoryprice.htm
Memory is getting cheap. Use it. 2011 $10/GB 2010 $20/GB
http://www.jcmit.com/memoryprice.htm
Memory is getting cheap. Use it. 2012 $5/GB 2011 $10/GB
2010 $20/GB http://www.jcmit.com/memoryprice.htm
Traditional Databases Have a Lot Of Features I Don’t Need
Locks & Latches
Transactions
Traditional Databases are Limited to Simple Data Access Key/Value Tabular
Traditional Databases are Limited to Simple Data Access Key/Value Tabular
(Boring)
Traditional Databases Are Not Real-Time
Basics of Behavioral Data
Actions
Actions & State
Actions & State + Time
Clickstream Logs Sensor Data Financial Transactions IVR
Important Differences In Behavioral Data
Behavioral Data is Historical
Behavioral Data is Isolated
Isolated = Concurrency
Performance / Internals
Aggregates 100M events / sec / core
Optimizations Stored by object then time Memory mapped Easy, compact
data format
Simplicity Supports Int64, Double, String & Bool MessagePack Encoded C99,
No Dependencies
Writing Your Own Language (Tangent)
EQL (Event Query Language)
EQL (Event Query Language)
Qip
Qip (Doesn’t stand for anything)
What is Qip? * LLVM-backed query processing language. * JIT
compiled on the fly. * As fast as C. * Removed from build because of complexity.
GitHub Archive Visualizer (Demo)
Ruby to Sky Integration
Add Event SkyDB.add_event( new Event( object_id:1, timestamp: Time.now, action: “/sign_up”,
data: { name: “John”, age: 20 } ) )
Next Actions SkyDB.next_actions( [ “/”, “/sign_up”, “/checkout” ] )
What’s Next For Sky?
More Analytics Functions!
Cohort Analysis 1 2 3 4 5 Jan 80% 70%
65% 63% 62% Feb 83% 73% 70% 69% Mar 87% 78% 75% Apr 89% 80% May 90% Month Signed Up Months After Signing Up
DAGs Home Page Sign Up Checkout View Product Cancel Order
Awesome Open Source Analytics Tools It’s like MixPanel that you
can install!
Modules!
Predictive Behavioral Analytics What will your users do next?
Risk Analysis
Anomaly Dectection / Fraud Detection
Questions?
Contact Info @benbjohnson
[email protected]