Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Instrumenting The Rest Of The Company: Hunting ...
Search
Eric Sigler
May 23, 2017
Technology
0
320
Instrumenting The Rest Of The Company: Hunting For Metrics
Presented at Monitorama 2017, video at:
https://youtu.be/wnjCNBfH3kg?t=3h3m35s
Eric Sigler
May 23, 2017
Tweet
Share
More Decks by Eric Sigler
See All by Eric Sigler
Four years of breaking things in production, on purpose.
esigler
0
52
A Brief Introduction To DevOps
esigler
0
97
Humans are terrible compilers: A User's Guide
esigler
0
110
Do You Know If Your Service Is Working Properly? A Guide To Being Paranoid.
esigler
0
160
"Is there any strong objection?"
esigler
0
210
Fear, Uncertainty, and Continuous Deployment
esigler
1
110
3AM, a survey.
esigler
0
210
Strategies For Being On Call & Keeping Your Sanity At The Same Time
esigler
0
160
Engineering for Engineers
esigler
0
83
Other Decks in Technology
See All in Technology
Google CloudのLLM活用の選択肢を広げるVertex AIのパートナーモデル
nayuts
0
130
2ヶ月かかるDBアップグレード検証を最大2週間に短縮した自作Go製CLIツール「Platinum」を紹介する / Introducing Go CLI tool "Platinum" for shortened DB upgrade validation
vtryo
2
120
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
5
46k
20240912 JJUGナイトセミナー
mii1004
0
140
OCI で始める!! Red Hat OpenShift / Get Started OpenShift on OCI
oracle4engineer
PRO
1
200
フルカイテン株式会社 採用資料
fullkaiten
0
32k
不動産 x AIことはじめ~データの真価を拓くために
estie
0
130
突撃! 隣のAmazon Bedrockユーザー 〜YouはどうしてAWSで?〜
minorun365
PRO
3
400
自作Cコンパイラ 8時間の奮闘
soukouki
0
860
株式会社EventHub・エンジニア採用資料
eventhub
0
3k
たった1人からはじめる【Agile Community of Practice】~ソース原理とFearless Changeを添えて~
ktc_corporate_it
1
510
AIで変わるテスト自動化:最新ツールの多様なアプローチ/ 20240910 Takahiro Kaneyama
shift_evolve
0
250
Featured
See All Featured
Building Your Own Lightsaber
phodgson
101
6k
Teambox: Starting and Learning
jrom
131
8.7k
What’s in a name? Adding method to the madness
productmarketing
PRO
21
3k
Why You Should Never Use an ORM
jnunemaker
PRO
53
9k
What's new in Ruby 2.0
geeforr
340
31k
GraphQLとの向き合い方2022年版
quramy
43
13k
4 Signs Your Business is Dying
shpigford
179
21k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
28
1.6k
Intergalactic Javascript Robots from Outer Space
tanoku
268
26k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Statistics for Hackers
jakevdp
794
220k
Bash Introduction
62gerente
608
210k
Transcript
@esigler Instrumenting The Rest Of The Company: Hunting For Useful
Metrics Eric Sigler, Head of DevOps, PagerDuty
@esigler Alternatively: ”Lies, Damn Lies, and Hacky Scripts"
@esigler
@esigler Engineer Eng Engineer Eng? Manager Mgr Manager
@esigler (No stock photos harmed in the making of this
talk.)
@esigler "We have problem $foo, so we're going to do
$bar."
@esigler "What data did you use to understand $foo? And
how will we know if $bar improved anything?”
@esigler “We can’t really measure either $foo and/or $bar.”
@esigler “Without data, you're just another person with an opinion.”
- W. Edwards Deming
@esigler
@esigler
@esigler
@esigler (Turns out other managers do this too.)
@esigler
@esigler "We have a problem with people not knowing what
the chatbot does, so we're going to write better documentation."
@esigler
@esigler ?
@esigler
@esigler “If only there was some way we could track
events, and show them over time.”
@esigler
@esigler
@esigler
@esigler Outcome: Writing a smarter help function in the chat
bot. (And simplifying some commands).
@esigler
@esigler Takeaway: Reuse existing tools when it makes sense.
@esigler
@esigler "We have slow tests in CI, so we're going
to complain a lot about it.”
@esigler “Define slow.”
@esigler Local != CI
@esigler
@esigler
@esigler
@esigler
@esigler “Tests take forever to start.”
@esigler
@esigler ?
@esigler
@esigler Outcome: More workers. (And, knowing how many to budget
for.)
@esigler Takeaway: Look for ways to reverse engineer existing metrics.
@esigler
@esigler "We have to ship code faster, so we're going
to reorganize."
@esigler
@esigler
@esigler But it doesn’t show where the bottlenecks are.
@esigler Pipe GitHub metrics into &
@esigler
@esigler
@esigler
@esigler Then start making changes.
@esigler
@esigler
@esigler Outcome: Productivity success! (With massive organizational change to enable
it.)
@esigler Takeaway: Look for proxy metrics
@esigler Potpourri: Data collection (chat, email, calendars) Cross-validation of metrics
(“Sniff test”) Cognitive biases around metrics Plotting against organization events
@esigler Takeaways: Useful metrics are everywhere You aren’t alone in
digging for metrics Existing tools can be repurposed Look to reverse engineer your way to a metric Look for proxy metrics (but choose wisely)
@esigler Thank you!
@esigler Image credits: https://commons.wikimedia.org/wiki/File:Staff_meeting.jpg https://blogs-images.forbes.com/kellyallan/files/2015/06/Deming-in-Tuxedo-DEM-1078-Dr.-Deming2-1940x1130.jpg (Wherever I grabbed that
screenshot from Pulp Fiction, my apologies I am a terrible person for not capturing the URL)