Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Instrumenting The Rest Of The Company: Hunting ...
Search
Eric Sigler
May 23, 2017
Technology
0
370
Instrumenting The Rest Of The Company: Hunting For Metrics
Presented at Monitorama 2017, video at:
https://youtu.be/wnjCNBfH3kg?t=3h3m35s
Eric Sigler
May 23, 2017
Tweet
Share
More Decks by Eric Sigler
See All by Eric Sigler
Four years of breaking things in production, on purpose.
esigler
0
58
A Brief Introduction To DevOps
esigler
0
110
Humans are terrible compilers: A User's Guide
esigler
0
120
Do You Know If Your Service Is Working Properly? A Guide To Being Paranoid.
esigler
0
180
"Is there any strong objection?"
esigler
0
230
Fear, Uncertainty, and Continuous Deployment
esigler
1
120
3AM, a survey.
esigler
0
230
Strategies For Being On Call & Keeping Your Sanity At The Same Time
esigler
0
160
Engineering for Engineers
esigler
0
92
Other Decks in Technology
See All in Technology
react-callを使ってダイヤログをいろんなとこで再利用しよう!
shinaps
2
260
OCI Oracle Database Services新機能アップデート(2025/06-2025/08)
oracle4engineer
PRO
0
180
機械学習を扱うプラットフォーム開発と運用事例
lycorptech_jp
PRO
0
600
現場で効くClaude Code ─ 最新動向と企業導入
takaakikakei
1
260
企業の生成AIガバナンスにおけるエージェントとセキュリティ
lycorptech_jp
PRO
2
190
2025/09/16 仕様駆動開発とAI-DLCが導くAI駆動開発の新フェーズ
masahiro_okamura
0
110
IoT x エッジAI - リアルタイ ムAI活用のPoCを今すぐ始め る方法 -
niizawat
0
110
「何となくテストする」を卒業するためにプロダクトが動く仕組みを理解しよう
kawabeaver
0
430
2つのフロントエンドと状態管理
mixi_engineers
PRO
3
120
5分でカオスエンジニアリングを分かった気になろう
pandayumi
0
260
[ JAWS-UG 東京 CommunityBuilders Night #2 ]SlackとAmazon Q Developerで 運用効率化を模索する
sh_fk2
3
460
自作JSエンジンに推しプロポーザルを実装したい!
sajikix
1
190
Featured
See All Featured
Making Projects Easy
brettharned
117
6.4k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.9k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
61k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
How to Think Like a Performance Engineer
csswizardry
26
1.9k
4 Signs Your Business is Dying
shpigford
184
22k
Mobile First: as difficult as doing things right
swwweet
224
9.9k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
30
9.7k
Transcript
@esigler Instrumenting The Rest Of The Company: Hunting For Useful
Metrics Eric Sigler, Head of DevOps, PagerDuty
@esigler Alternatively: ”Lies, Damn Lies, and Hacky Scripts"
@esigler
@esigler Engineer Eng Engineer Eng? Manager Mgr Manager
@esigler (No stock photos harmed in the making of this
talk.)
@esigler "We have problem $foo, so we're going to do
$bar."
@esigler "What data did you use to understand $foo? And
how will we know if $bar improved anything?”
@esigler “We can’t really measure either $foo and/or $bar.”
@esigler “Without data, you're just another person with an opinion.”
- W. Edwards Deming
@esigler
@esigler
@esigler
@esigler (Turns out other managers do this too.)
@esigler
@esigler "We have a problem with people not knowing what
the chatbot does, so we're going to write better documentation."
@esigler
@esigler ?
@esigler
@esigler “If only there was some way we could track
events, and show them over time.”
@esigler
@esigler
@esigler
@esigler Outcome: Writing a smarter help function in the chat
bot. (And simplifying some commands).
@esigler
@esigler Takeaway: Reuse existing tools when it makes sense.
@esigler
@esigler "We have slow tests in CI, so we're going
to complain a lot about it.”
@esigler “Define slow.”
@esigler Local != CI
@esigler
@esigler
@esigler
@esigler
@esigler “Tests take forever to start.”
@esigler
@esigler ?
@esigler
@esigler Outcome: More workers. (And, knowing how many to budget
for.)
@esigler Takeaway: Look for ways to reverse engineer existing metrics.
@esigler
@esigler "We have to ship code faster, so we're going
to reorganize."
@esigler
@esigler
@esigler But it doesn’t show where the bottlenecks are.
@esigler Pipe GitHub metrics into &
@esigler
@esigler
@esigler
@esigler Then start making changes.
@esigler
@esigler
@esigler Outcome: Productivity success! (With massive organizational change to enable
it.)
@esigler Takeaway: Look for proxy metrics
@esigler Potpourri: Data collection (chat, email, calendars) Cross-validation of metrics
(“Sniff test”) Cognitive biases around metrics Plotting against organization events
@esigler Takeaways: Useful metrics are everywhere You aren’t alone in
digging for metrics Existing tools can be repurposed Look to reverse engineer your way to a metric Look for proxy metrics (but choose wisely)
@esigler Thank you!
@esigler Image credits: https://commons.wikimedia.org/wiki/File:Staff_meeting.jpg https://blogs-images.forbes.com/kellyallan/files/2015/06/Deming-in-Tuxedo-DEM-1078-Dr.-Deming2-1940x1130.jpg (Wherever I grabbed that
screenshot from Pulp Fiction, my apologies I am a terrible person for not capturing the URL)