Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
MongoDB for Analytics
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
John Nunemaker
PRO
October 18, 2011
Programming
30k
16
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
MongoDB for Analytics
Presented at Mongo Chicago 2011.
John Nunemaker
PRO
October 18, 2011
Transcript
50 slides
Ordered List John Nunemaker MongoChi 2011 October 18, 2011 MongoDB
for Analytics A loving conversation with @jnunemaker
Background As presented through interpretive dance
None
None
None
~1 month Of evenings and weekends
~4 dog years Since public launch
~6 tiny servers 2 web, 2 app, 2 db
~1-2 Million Page views per day
None
None
Implementation Imma show you how we do what we do
baby
Doing It Live No aggregate querying
get('/track.gif') do Hit.record(...) TrackGif end
class Hit def record site.atomic_update(site_updates) Resolution.record(self) Technology.record(self) Location.record(self) Referrer.record(self) Content.record(self)
Search.record(self) Notification.record(self) View.record(self) end end
class Resolution def record(hit) query = {'_id' => "..."} update
= {'$inc' => {}} update['$inc']["sx.#{hit.screenx}"] = 1 update['$inc']["bx.#{hit.browserx}"] = 1 update['$inc']["by.#{hit.browsery}"] = 1 collection(hit.created_on) .update(query, update, :upsert => true) end end end
Pros
Pros Space
Pros Space RAM
Pros Space RAM Reads
Pros Space RAM Reads Live
Cons
Cons Writes
Cons Writes Constraints
Cons Writes Constraints More Forethought
Cons Writes Constraints More Forethought No raw data
Time Frame Minute, hour, month, day, year, forever?
# of Variations One document vs many
Single Document Per Time Frame
None
{ "t" => 336381, "u" => 158951, "2011" => {
"02" => { "18" => { "t" => 9, "u" => 6 } } } }
{ '$inc' => { 't' => 1, 'u' => 1,
'2011.02.18.t' => 1, '2011.02.18.u' => 1, } }
Single Document For all ranges in time frame
None
{ "_id" =>"...:10", "bx" => { "320" => 85, "480"
=> 318, "800" => 1938, "1024" => 5033, "1280" => 6288, "1440" => 2323, "1600" => 3817, "2000" => 137 }, "by" => { "480" => 2205, "600" => 7359,
"600" => 7359, "768" => 4515, "900" => 3833, "1024"
=> 2026 }, "sx" => { "320" => 191, "480" => 179, "800" => 195, "1024" => 1059, "1280" => 5861, "1440" => 3533, "1600" => 7675, "2000" => 1279 } }
{ '$inc' => { 'sx.1440' => 1, 'bx.1280' => 1,
'by.768' => 1, } }
Many Documents Search terms, content, referrers...
None
[ { "_id" => "<oid>:<hash>", "t" => "ruby class variables",
"sid" => BSON::ObjectId('<oid>'), "v" => 352 }, { "_id" => "<oid>:<hash>", "t" => "ruby unless", "sid" => BSON::ObjectId('<oid>'), "v" => 347 }, ]
Writes {'_id' => "#{site_id}:#{hash}"}
Reads [['sid', 1], ['v', -1]]
Growth The best laid plans of mice and men
Partition Hot Data Currently using collections for time frames
Bigger, Faster Server More CPU, RAM, Disk Space
Users Sites Content Referrers Terms Engines Resolutions Locations Users Sites
Content Referrers Terms Engines Resolutions Locations
Partition by Function Spread writes across a few servers
Users Sites Content Referrers Terms Engines Resolutions Locations
Partition by Server Spread writes across a ton of servers,
way down the road, not worried yet
Ordered List Thank you!
[email protected]
John Nunemaker MongoChi 2011 October
18, 2011 @jnunemaker
More Decks by John Nunemaker
See All by John Nunemaker
Remote First: Building Distributed Teams that Win
jnunemaker
PRO
1
240
AI: The stuff that nobody shows you
jnunemaker
PRO
12
1.3k
Atom
jnunemaker
PRO
10
5.3k
MongoDB for Analytics
jnunemaker
PRO
11
1.2k
Addicted to Stable
jnunemaker
PRO
32
2.9k
MongoDB for Analytics
jnunemaker
PRO
21
2.4k
Why You Should Never Use an ORM
jnunemaker
PRO
61
10k
Why NoSQL?
jnunemaker
PRO
10
1.1k
Don't Repeat Yourself, Repeat Others
jnunemaker
PRO
7
3.6k
Other Decks in Programming
See All in Programming
protojson における文字化けの扱い
kazuki1023
0
460
Kotlin JVM用に作られたライブラリのKMPを前提とした再設計
jsoizo
0
220
Kawaiicon: Kubernetes, it's no longer cool, but let's secure it anyway!
barnbarn
0
140
正解の無い問題を 「できるだけ上手く」解く ヒューリスティックで感性を磨こう。〜現実課題をプログラミングで解きあうコミュニティについて〜
allegrogiken
0
110
読みやすい!と評判(?)の発表の準備にかける私のこだわり
shoheimitani
0
130
React Native経験者が Flutterを30⽇間触って驚いたこと
3saito
0
200
Web3Dライブラリを作ってVue Custom Rendererの仕組みを理解しよう / lets learn vue custom renderer by making 3d library
drumath2237
1
140
TanStack Vue Query と Pinia Colada から学ぶ非同期状態の型設計
aoshi
0
700
小さなアプリを作って公開するまで
onozaty
0
220
DroidKnights 2026 Android A2UI Jetpack Compose 기반의 안전한 AI UI 렌더링 - 박제창
itsmedreamwalker
1
260
Getting Started with Vibe Coding
miccheng
0
250
AI Augmentation Techniques with Liberty
ivargrimstad
1
540
Featured
See All Featured
Raft: Consensus for Rubyists
vanstee
143
7.9k
So, you think you're a good person
axbom
PRO
2
2.3k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
47
8.4k
Mozcon NYC 2025: Stop Losing SEO Traffic
samtorres
1
610
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
37
6.7k
From Legacy to Launchpad: Building Startup-Ready Communities
dugsong
0
380
Visualization
eitanlees
153
17k
Become a Pro
speakerdeck
PRO
31
6.4k
Why Mistakes Are the Best Teachers: Turning Failure into a Pathway for Growth
auna
0
360
Heart Work Chapter 1 - Part 1
lfama
PRO
11
37k
Being A Developer After 40
akosma
PRO
92
590k
Paper Plane (Part 1)
katiecoart
PRO
3
12k