Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Behavioral Databases
Search
benbjohnson
October 24, 2012
Technology
16
2.5k
Behavioral Databases
A presentation I gave at the Derailed meetup in Denver on October 24th, 2012.
benbjohnson
October 24, 2012
Tweet
Share
More Decks by benbjohnson
See All by benbjohnson
Why You Shouldn't Write a Database
benbjohnson
2
550
Static Code Analysis Using SSA
benbjohnson
6
1.7k
Writing a High Performance Database in Go
benbjohnson
11
6.8k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
106
28k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
2k
New Tech - Sky
benbjohnson
2
380
Sky Analytics Pitch Deck
benbjohnson
0
600
Other Decks in Technology
See All in Technology
ホリスティックテスティングの右側も大切にする 〜2つの[はか]る〜 / Holistic Testing: Right Side Matters
nihonbuson
PRO
0
580
人に寄り添うAIエージェントとアーキテクチャ #BetAIDay
layerx
PRO
8
2k
LLMをツールからプラットフォームへ〜Ai Workforceの戦略〜 #BetAIDay
layerx
PRO
1
860
LIFF CLIとngrokを使ったLIFF/LINEミニアプリのお手軽実機確認
diggymo
0
230
【OptimizationNight】数理最適化のラストワンマイルとしてのUIUX
brainpadpr
1
260
AI コードレビューが面倒すぎるのでテスト駆動開発で解決しようとして読んだら、根本的に俺の勘違いだった
mutsumix
0
160
AI によるドキュメント処理を加速するためのOCR 結果の永続化と再利用戦略
tomoaki25
0
400
LTに影響を受けてテンプレリポジトリを作った話
hol1kgmg
0
290
Unson OS|48時間で「売れるか」を判定する AI 市場検証プラットフォーム
unson
0
170
反脆弱性(アンチフラジャイル)とデータ基盤構築
cuebic9bic
2
160
マルチプロダクト×マルチテナントを支えるモジュラモノリスを中心としたアソビューのアーキテクチャ
disc99
0
310
2025-07-31: GitHub Copilot Agent mode at Vibe Coding Cafe (15min)
chomado
2
380
Featured
See All Featured
Facilitating Awesome Meetings
lara
54
6.5k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
6k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
50
5.5k
Agile that works and the tools we love
rasmusluckow
329
21k
The Cult of Friendly URLs
andyhume
79
6.5k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Building an army of robots
kneath
306
45k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.8k
Documentation Writing (for coders)
carmenintech
73
5k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
Transcript
Behavioral Databases Next Generation NoSQL Analytics By Ben Johnson
My Background
Former Oracle DBA
Former Oracle DBA Behavioral Analytics
Former Oracle DBA Data Visualization Behavioral Analytics
Why did I write a database?
Other Databases Are Too Slow Redis 100K ops/second PostgreSQL 5K
QPS/core
Processing Needed to Occur Near Data Client Server 1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms 1ms 1ms
Processing Needed to Occur Near Data Client Server 1ms 1ms
1ms 1ms 1ms 1ms 1ms 1ms
Processing Needed to Occur Near Data
Memory is getting cheap. Use it. 2010 $20/GB http://www.jcmit.com/memoryprice.htm
Memory is getting cheap. Use it. 2011 $10/GB 2010 $20/GB
http://www.jcmit.com/memoryprice.htm
Memory is getting cheap. Use it. 2012 $5/GB 2011 $10/GB
2010 $20/GB http://www.jcmit.com/memoryprice.htm
Traditional Databases Have a Lot Of Features I Don’t Need
Locks & Latches
Transactions
Traditional Databases are Limited to Simple Data Access Key/Value Tabular
Traditional Databases are Limited to Simple Data Access Key/Value Tabular
(Boring)
Traditional Databases Are Not Real-Time
Basics of Behavioral Data
Actions
Actions & State
Actions & State + Time
Clickstream Logs Sensor Data Financial Transactions IVR
Important Differences In Behavioral Data
Behavioral Data is Historical
Behavioral Data is Isolated
Isolated = Concurrency
Performance / Internals
Aggregates 100M events / sec / core
Optimizations Stored by object then time Memory mapped Easy, compact
data format
Simplicity Supports Int64, Double, String & Bool MessagePack Encoded C99,
No Dependencies
Writing Your Own Language (Tangent)
EQL (Event Query Language)
EQL (Event Query Language)
Qip
Qip (Doesn’t stand for anything)
What is Qip? * LLVM-backed query processing language. * JIT
compiled on the fly. * As fast as C. * Removed from build because of complexity.
GitHub Archive Visualizer (Demo)
Ruby to Sky Integration
Add Event SkyDB.add_event( new Event( object_id:1, timestamp: Time.now, action: “/sign_up”,
data: { name: “John”, age: 20 } ) )
Next Actions SkyDB.next_actions( [ “/”, “/sign_up”, “/checkout” ] )
What’s Next For Sky?
More Analytics Functions!
Cohort Analysis 1 2 3 4 5 Jan 80% 70%
65% 63% 62% Feb 83% 73% 70% 69% Mar 87% 78% 75% Apr 89% 80% May 90% Month Signed Up Months After Signing Up
DAGs Home Page Sign Up Checkout View Product Cancel Order
Awesome Open Source Analytics Tools It’s like MixPanel that you
can install!
Modules!
Predictive Behavioral Analytics What will your users do next?
Risk Analysis
Anomaly Dectection / Fraud Detection
Questions?
Contact Info @benbjohnson
[email protected]