Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Data Visualization in the Trenches
Search
Abe Stanway
May 17, 2013
Programming
5
720
Data Visualization in the Trenches
This talk was given at Bocoup's OpenVis Conf in Boston.
Abe Stanway
May 17, 2013
Tweet
Share
More Decks by Abe Stanway
See All by Abe Stanway
Building Data Driven Organizations
astanway
1
210
MOM! My algorithms SUCK
astanway
15
2.8k
A Deep Dive into Monitoring with Skyline
astanway
6
1.8k
Bring the Noise: Continuously Deploying Under a Hailstorm of Metrics
astanway
34
8k
Gifs as Language
astanway
2
850
Your API is a Product
astanway
3
980
Zen and the Art of Writing Commit Logs
astanway
3
830
Other Decks in Programming
See All in Programming
開発生産性を上げるための生成AI活用術
starfish719
3
1.2k
bootcamp2025_バックエンド研修_WebAPIサーバ作成.pdf
geniee_inc
0
110
Leading Effective Engineering Teams in the AI Era
addyosmani
7
450
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
190
チームの境界をブチ抜いていけ
tokai235
0
180
Claude CodeによるAI駆動開発の実践 〜そこから見えてきたこれからのプログラミング〜
iriikeita
0
280
ソフトウェア設計の実践的な考え方
masuda220
PRO
4
600
Domain-centric? Why Hexagonal, Onion, and Clean Architecture Are Answers to the Wrong Question
olivergierke
3
900
Railsだからできる 例外業務に禍根を残さない 設定設計パターン
ei_ei_eiichi
0
930
Foundation Modelsを実装日本語学習アプリを作ってみた!
hypebeans
0
120
なぜあの開発者はDevRelに伴走し続けるのか / Why Does That Developer Keep Running Alongside DevRel?
nrslib
3
410
私はどうやって技術力を上げたのか
yusukebe
44
19k
Featured
See All Featured
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
16
1.7k
Git: the NoSQL Database
bkeepers
PRO
431
66k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Code Reviewing Like a Champion
maltzj
526
40k
Producing Creativity
orderedlist
PRO
347
40k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
35
6.1k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
2.5k
What's in a price? How to price your products and services
michaelherold
246
12k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
285
14k
RailsConf 2023
tenderlove
30
1.2k
Transcript
Abe Stanway @abestanway Data visualization in the trenches
None
1.5 Billion page views $117 Million of goods sold 950
thousand users
1.5 Billion page views $117 Million of goods sold 950
thousand users (in December)
Text We practice continuous deployment.
de • ploy /diˈploi/ Verb To release your code for
the world to see, hopefully without breaking the Internet
250+ committers, everyone deploys.
Day one: Deploy.
None
30+ DEPLOYS A DAY
Text “30 deploys a day? Is that safe?”
Text Yes, with the proper tooling.
Text Every engineer must have a finger on the pulse
of the system.
Text How do you make an entire web stack “consumable”
to a handful of engineers?
Text More information Quickly consumable More abstraction
p
None
Text Real time error logging
Text No abstraction. Fluffy information. Easy to consume.
“Not all things that break throw errors.” - Oscar Wilde
1. ssh to server 2. poke around for the log
files 3. try to remember what they mean 4. try to scroll back in time to find when they started acting up. 5. repeat
Text No abstraction. Fluffy information.
Text ...but hard to get at. Lots of friction means
not easily consumable.
Text Bump up a layer of abstraction.
1. create a graph 2. look at the graph 3.
?? 4. profit!!
StatsD
StatsD::increment(“foo.bar”)
If it moves, graph it!
If it doesn’t move, graph it anyway (it might make
a run for it)
None
Text Some abstraction. Denser information.
Text ...still hard to get at en masse.
Text Bump up a layer of abstraction.
DASHBOARDS!
Hang out with the dashboards after you push.
None
[1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 20]
[1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 60]
DASHBOARDS x 250000 !
None
“...but there are also unknown unknowns - there are things
we do not know we don’t know.”
Text Slightly denser information is negated by the deluge.
Text The majority remains unconsumable.
Text Bump up a layer of abstraction.
SKYLINE
A real time anomaly detection system
None
Text Very abstract, harder to understand, but the effective information
density is massive.
Text Consumption is also increased by outsourcing it to the
machine.
Text Trust becomes an issue.
So you found an anomaly.
MAYBE THERE ARE OTHERS
Oculus: a metrics correlation system
None
Text A good tool adds “touch” to the system.
Text More “touch” means more intimacy with the stack.
Text A delicate balance between insight, consumptive capacity, and actionability.
Text More abstraction leads to less intimacy...
Text ...but greater information density.
@abestanway
[email protected]
Abe Stanway Data Engineer Thanks!