Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
You, latency and profiling @ GolangUK 2017
Search
Filippo Valsorda
August 17, 2017
Programming
0
1.2k
You, latency and profiling @ GolangUK 2017
Filippo Valsorda
August 17, 2017
Tweet
Share
More Decks by Filippo Valsorda
See All by Filippo Valsorda
Asynchronous networking @ GopherCon 2018
filosottile
3
2.1k
Le note cifrate di Antonio Marzi
filosottile
1
400
Why cgo is slow @ CapitalGo 2018
filosottile
2
4.6k
Squeezing a key through a carry bit @ 34c3
filosottile
0
1.7k
Calling Rust from Go, without cgo @ GothamGo 2017
filosottile
1
2.7k
Encrypting the Internet with Go @ GopherCon 2017
filosottile
9
2.6k
You, latency and profiling @ GopherCon India 2017
filosottile
13
4.1k
TLS 1.3 @ 33c3
filosottile
4
6.9k
Stealing Bitcoin With Math - HOPE XI
filosottile
13
30k
Other Decks in Programming
See All in Programming
DockerからECSへ 〜 AWSの海に出る前に知っておきたいこと 〜
ota1022
5
1.8k
GitHub Copilotの全体像と活用のヒント AI駆動開発の最初の一歩
74th
8
3.3k
Oracle Database Technology Night 92 Database Connection control FAN-AC
oracle4engineer
PRO
1
280
MCPで実現するAIエージェント駆動のNext.jsアプリデバッグ手法
nyatinte
7
960
令和最新版手のひらコンピュータ
koba789
14
8.1k
ゲームの物理
fadis
5
1.6k
Understanding Ruby Grammar Through Conflicts
yui_knk
1
140
私の後悔をAWS DMSで解決した話
hiramax
4
170
Namespace and Its Future
tagomoris
6
630
開発チーム・開発組織の設計改善スキルの向上
masuda220
PRO
17
9.1k
MLH State of the League: 2026 Season
theycallmeswift
0
170
自作OSでDOOMを動かしてみた
zakki0925224
1
1.4k
Featured
See All Featured
A designer walks into a library…
pauljervisheath
207
24k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
30
9.6k
Imperfection Machines: The Place of Print at Facebook
scottboms
268
13k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Done Done
chrislema
185
16k
Building a Modern Day E-commerce SEO Strategy
aleyda
43
7.5k
Building an army of robots
kneath
306
46k
GraphQLとの向き合い方2022年版
quramy
49
14k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
139
34k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
Mobile First: as difficult as doing things right
swwweet
224
9.9k
Transcript
You, latency and profiling Filippo Valsorda — @FiloSo/le
What is fast?
A fast hash… … can hash many MB per second
A fast regex… … can process many MB/s
A fast database… … can store many GB per second
A fast database… … can store many GB per second
… or replies to queries in a few ms
A fast website… … can handle many requests … or
loads in a few milliseconds
A fast API… … can handle many clients … or
answers in a few milliseconds
Fast is… Throughput and Latency
CPU profiling -cpuprofile /debug/pprof/profile
SIGPROF CPU profiling
runtime/proc.go
h>p.Handler 1 Running on CPU h>p.Handler 2 I/O I/O h>p.Handler
3 I/O h>p.Handler 4 I/O CPU profiling
Running on CPU I/O I/O I/O I/O h>p.Handler 1 h>p.Handler
2 h>p.Handler 3 h>p.Handler 4 CPU profiling
I/O Observed latency h>p.Handler 1 CPU profiling
func Write(data []byte) { for i := 0; i <
50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } } func Hash(data []byte) { for i := 0; i < 50; i++ { sha256.Sum256(data) } }
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total go tool pprof -web
slowserver cpu.pprof
None
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total $ time
curl http://127.0.0.1:12345/write-no-hash 7.692 total
Latency is not all about CPU
CPU profiling is for throughput The tracer is for latency
$ time curl http://127.0.0.1:12345/write-no-hash 7.692 total go tool trace -pprof=syscall
trace.out > syscall.pprof go tool pprof -web syscall.pprof
None
The tracer Detailed nanosecond-level log of execuOon events: • gorouOne
scheduling, blocking • syscalls, network, I/O • garbage collecOon
•log files are BIG •25% overhead (aWer Go 1.7) •symbols
are embedded (Go 1.7) The tracer
•Full event context •GorouOne metadata and history •Discrete events, not
samples Trace vs. profile
CPU profiling Total: 20 samples
Tracing events
import "runtime/trace" import "net/http/pprof" /debug/pprof/trace?seconds=5 The tracer
None
go tool trace -pprof=TYPE trace.out Supported profile types are: -
net: network blocking profile - sync: synchronization blocking profile - syscall: syscall blocking profile - sched: scheduler latency profile
go tool trace -pprof=syscall trace.out > syscall.pprof go tool pprof
-web syscall.pprof
go tool trace -pprof=syscall trace.out > syscall.pprof go-torch -b syscall.pprof
&& open torch.svg
None
None
-pprof=syscall func Write(data []byte) { for i := 0; i
< 50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } }
-pprof=sync // Block waits on a channel func Block() {
<-time.After(1 * time.Second) }
-pprof=sync
-pprof=sync
-pprof=net func Download() []byte { r, err := http.Get("https://golang.org/doc/go1.8") body,
err := ioutil.ReadAll(r.Body) r.Body.Close() return body }
-pprof=net
-pprof=net
-pprof=net
None
WriIng our own profile prof := make(map[uint64]pprof.Record) for _, ev
:= range events { if ev.Type != trace.EvGoBlockNet { continue } rec := prof[ev.StkID] rec.Stk = ev.Stk rec.N++ rec.Time += ev.Link.Ts - ev.Ts prof[ev.StkID] = rec } pprof.BuildProfile(prof).Write(os.Stdout)
var childG = make(map[uint64]struct{}) var lastGLen int for { for
_, ev := range events { if ev.Type != trace.EvGoCreate { continue } if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) { continue } childG[ev.Args[0]] = struct{}{} } if len(childG) == lastGLen { break } lastGLen = len(childG) }
tracefocus tracefocus -filter=ServeHTTP trace.out > net.pprof github.com/FiloSottile/tracetools
None
None
Build more tools! • Focus on gorouOne number • Aggregate
all blocking types • VisualizaOons • …
tracehist tracehist -filter=Close trace.out github.com/FiloSottile/tracetools
None
Build more tools! Profile latency!
QuesIons? Filippo Valsorda @FiloSo/le hi@filippo.io