Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
You, latency and profiling @ GolangUK 2017
Search
Filippo Valsorda
August 17, 2017
Programming
1.3k
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
You, latency and profiling @ GolangUK 2017
Filippo Valsorda
August 17, 2017
Transcript
54 slides
You, latency and profiling Filippo Valsorda — @FiloSo/le
What is fast?
A fast hash… … can hash many MB per second
A fast regex… … can process many MB/s
A fast database… … can store many GB per second
A fast database… … can store many GB per second
… or replies to queries in a few ms
A fast website… … can handle many requests … or
loads in a few milliseconds
A fast API… … can handle many clients … or
answers in a few milliseconds
Fast is… Throughput and Latency
CPU profiling -cpuprofile /debug/pprof/profile
SIGPROF CPU profiling
runtime/proc.go
h>p.Handler 1 Running on CPU h>p.Handler 2 I/O I/O h>p.Handler
3 I/O h>p.Handler 4 I/O CPU profiling
Running on CPU I/O I/O I/O I/O h>p.Handler 1 h>p.Handler
2 h>p.Handler 3 h>p.Handler 4 CPU profiling
I/O Observed latency h>p.Handler 1 CPU profiling
func Write(data []byte) { for i := 0; i <
50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } } func Hash(data []byte) { for i := 0; i < 50; i++ { sha256.Sum256(data) } }
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total go tool pprof -web
slowserver cpu.pprof
None
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total $ time
curl http://127.0.0.1:12345/write-no-hash 7.692 total
Latency is not all about CPU
CPU profiling is for throughput The tracer is for latency
$ time curl http://127.0.0.1:12345/write-no-hash 7.692 total go tool trace -pprof=syscall
trace.out > syscall.pprof go tool pprof -web syscall.pprof
None
The tracer Detailed nanosecond-level log of execuOon events: • gorouOne
scheduling, blocking • syscalls, network, I/O • garbage collecOon
•log files are BIG •25% overhead (aWer Go 1.7) •symbols
are embedded (Go 1.7) The tracer
•Full event context •GorouOne metadata and history •Discrete events, not
samples Trace vs. profile
CPU profiling Total: 20 samples
Tracing events
import "runtime/trace" import "net/http/pprof" /debug/pprof/trace?seconds=5 The tracer
None
go tool trace -pprof=TYPE trace.out Supported profile types are: -
net: network blocking profile - sync: synchronization blocking profile - syscall: syscall blocking profile - sched: scheduler latency profile
go tool trace -pprof=syscall trace.out > syscall.pprof go tool pprof
-web syscall.pprof
go tool trace -pprof=syscall trace.out > syscall.pprof go-torch -b syscall.pprof
&& open torch.svg
None
None
-pprof=syscall func Write(data []byte) { for i := 0; i
< 50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } }
-pprof=sync // Block waits on a channel func Block() {
<-time.After(1 * time.Second) }
-pprof=sync
-pprof=sync
-pprof=net func Download() []byte { r, err := http.Get("https://golang.org/doc/go1.8") body,
err := ioutil.ReadAll(r.Body) r.Body.Close() return body }
-pprof=net
-pprof=net
-pprof=net
None
WriIng our own profile prof := make(map[uint64]pprof.Record) for _, ev
:= range events { if ev.Type != trace.EvGoBlockNet { continue } rec := prof[ev.StkID] rec.Stk = ev.Stk rec.N++ rec.Time += ev.Link.Ts - ev.Ts prof[ev.StkID] = rec } pprof.BuildProfile(prof).Write(os.Stdout)
var childG = make(map[uint64]struct{}) var lastGLen int for { for
_, ev := range events { if ev.Type != trace.EvGoCreate { continue } if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) { continue } childG[ev.Args[0]] = struct{}{} } if len(childG) == lastGLen { break } lastGLen = len(childG) }
tracefocus tracefocus -filter=ServeHTTP trace.out > net.pprof github.com/FiloSottile/tracetools
None
None
Build more tools! • Focus on gorouOne number • Aggregate
all blocking types • VisualizaOons • …
tracehist tracehist -filter=Close trace.out github.com/FiloSottile/tracetools
None
Build more tools! Profile latency!
QuesIons? Filippo Valsorda @FiloSo/le hi@filippo.io
More Decks by Filippo Valsorda
See All by Filippo Valsorda
Asynchronous networking @ GopherCon 2018
filosottile
3
2.2k
Le note cifrate di Antonio Marzi
filosottile
1
470
Why cgo is slow @ CapitalGo 2018
filosottile
2
5k
Squeezing a key through a carry bit @ 34c3
filosottile
0
1.9k
Calling Rust from Go, without cgo @ GothamGo 2017
filosottile
1
3k
Encrypting the Internet with Go @ GopherCon 2017
filosottile
9
2.8k
You, latency and profiling @ GopherCon India 2017
filosottile
13
4.4k
TLS 1.3 @ 33c3
filosottile
4
7.1k
Stealing Bitcoin With Math - HOPE XI
filosottile
13
30k
Other Decks in Programming
See All in Programming
JavaScriptはシングルバイナリの夢を見るか / Does JavaScript Dream of a Single Binary?
ssssota
0
120
Enforcing Your Architecture with Unit Tests in the AI Era (Swift Connection X)
steliosf
PRO
1
190
Beyond Chatbots: Agentic UI with Open Standards @W-JAX 2026, München
manfredsteyer
PRO
0
130
protojson における文字化けの扱い
kazuki1023
0
460
AWS CDKを理解するために公式の定義を深掘って解説してみる / Understanding AWS CDK by Digging into Its Official Definition
akihisaikeda
3
330
Debian JP Projectのサーバー構成
uwabami
1
170
SODA - FACT BOOK(EN)
sodainc
1
280
Claude Code ハーネスDeep Dive – 君はどこまで制御する? -
takateeen_knj
6
6.3k
仕様駆動開発入門 (何を作るかを先に決めて渡す) | 目指せ Kiro マスター 05
inariku
PRO
3
220
ビジネスロジックを堅牢に~Scala 3のドメインプリミティブ~
j5ik2o
0
230
SONY CISC-NEWS NWS-1750 + NWB-225 ハードウェアメンテとフレームバッファカラー対応 / KOF2026
tsutsui
0
130
数行の変更でDockerビルドを最大91%高速化~GitLab CIの実行基盤を社内40リポジトリに展開するまで~
picopico
1
130
Featured
See All Featured
Lightning Talk: Beautiful Slides for Beginners
inesmontani
PRO
2
860
Code Reviewing Like a Champion
maltzj
529
40k
We Analyzed 250 Million AI Search Results: Here's What I Found
joshbly
2
2.1k
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
akashhashmi
0
560
jQuery: Nuts, Bolts and Bling
dougneiner
66
8.7k
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
uxyall
2
540
Understanding Cognitive Biases in Performance Measurement
bluesmoon
32
3.1k
The agentic SEO stack - context over prompts
schlessera
0
1k
Side Projects
sachag
456
43k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Winning Ecommerce Organic Search in an AI Era - #searchnstuff2025
aleyda
2
2.2k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
10k