Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
RPC Metrics at Google
Search
JBD
August 09, 2018
Programming
2
500
RPC Metrics at Google
JBD
August 09, 2018
Tweet
Share
More Decks by JBD
See All by JBD
eBPF in Microservices Observability at eBPF Day
rakyll
1
2k
eBPF in Microservices Observability
rakyll
1
1.6k
OpenTelemetry at AWS
rakyll
1
1.8k
Debugging Code Generation in Go
rakyll
5
1.5k
Are you ready for production?
rakyll
8
2.6k
Servers are doomed to fail
rakyll
3
1.5k
Serverless Containers
rakyll
1
240
Critical Path Analysis
rakyll
0
500
Monitoring and Debugging Containers
rakyll
2
1.1k
Other Decks in Programming
See All in Programming
unique パッケージから学ぶ interning と weak reference @ Asakusa.go#3
karamaru
2
810
What is Parser
yui_knk
9
4.1k
ドメイン駆動設計を実践するために必要なもの
bikisuke
4
330
Ebitengineの1vs1ゲーム WebRTCの活用
ponyo877
0
380
あなたのアプリ、ログはでてますか?あるいはログをだしてますか? (Funabashi.dev用 軽量版)
uzulla
2
120
いつか使える ObjectSpace / Maybe useful ObjectSpace
euglena1215
2
140
大公開!iOS開発の悩みトップ5 〜iOSDC Japan 2024〜
ryunakayama
0
190
Method Swizzlingを行うライブラリにおけるマルチモジュール設計
yoshikma
0
120
事業フェーズの変化に対応する 開発生産性向上のゼロイチ
masaygggg
0
200
Our Websites Need a Lifestyle Change, Not a Diet
ryantownsend
0
150
Some more adventure of Happy Eyeballs
coe401_
2
190
React + TextAliveでカッコいいLyric Applicatioinを作ろう!!
tosuri13
0
400
Featured
See All Featured
Keith and Marios Guide to Fast Websites
keithpitt
408
22k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
278
13k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
1
53
How GitHub (no longer) Works
holman
310
140k
Bootstrapping a Software Product
garrettdimon
PRO
304
110k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
36
1.7k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
326
21k
KATA
mclloyd
27
13k
How to Think Like a Performance Engineer
csswizardry
16
960
Put a Button on it: Removing Barriers to Going Fast.
kastner
58
3.4k
The Pragmatic Product Professional
lauravandoore
31
6.2k
Designing for humans not robots
tammielis
248
25k
Transcript
RPC Metrics at Google JBD, Google (@rakyll)
gRPC Metrics at Google JBD, Google (@rakyll)
Request Metrics at Google JBD, Google (@rakyll)
@rakyll "100% is the wrong reliability target for basically everything."
-- Benjamin Treynor Sloss, VP of Engineering, Google
@rakyll "A service is available if users cannot tell that
there was an outage."
@rakyll Principled way of saying what level of downtime is
acceptable. • Error rate • Latency expectations SLOs
@rakyll Analytics frontend server Authentication Reporting Users ... Spanner Blob
Store
@rakyll Questions infra teams want to ask: • Are we
meeting the SLO for the other team? • What’s the impact of a product on infra? • How much do we need to scale up if product grows 10%?
@rakyll High-Cardinality Breaking down the metrics data...
@rakyll Query the collected data in various ways: • Latency
distribution for RPCs originated at Google Analytics. • Requests take took more than 100ms for the customer #123. • Compare the request latency initiated at web vs mobile frontend.
@rakyll Analytics frontend server Authentication Reporting Users ... Spanner Blob
Store originator=analytics; ...
@rakyll Blob store read errors by originator
@rakyll Dynamically choose aggregation (split between recording and aggregation)
@rakyll Exemplars
@rakyll /rpz and /statz
@rakyll http://server:7777/debug/rpcz
@rakyll Export? Monarch, Prometheus, and more.
@rakyll import “cloud.google.com/go/pubsub”
@rakyll +
Thank you! JBD, Google
[email protected]
@rakyll