Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
You, latency and profiling @ GolangUK 2017
Search
Filippo Valsorda
August 17, 2017
Programming
0
1.1k
You, latency and profiling @ GolangUK 2017
Filippo Valsorda
August 17, 2017
Tweet
Share
More Decks by Filippo Valsorda
See All by Filippo Valsorda
Asynchronous networking @ GopherCon 2018
filosottile
3
1.9k
Le note cifrate di Antonio Marzi
filosottile
1
340
Why cgo is slow @ CapitalGo 2018
filosottile
2
4.3k
Squeezing a key through a carry bit @ 34c3
filosottile
0
1.5k
Calling Rust from Go, without cgo @ GothamGo 2017
filosottile
1
2.5k
Encrypting the Internet with Go @ GopherCon 2017
filosottile
9
2.6k
You, latency and profiling @ GopherCon India 2017
filosottile
13
3.9k
TLS 1.3 @ 33c3
filosottile
4
6.6k
Stealing Bitcoin With Math - HOPE XI
filosottile
13
29k
Other Decks in Programming
See All in Programming
What’s New in Compose Multiplatform - A Live Tour (droidcon London 2024)
zsmb
1
340
レガシーな Android アプリのリアーキテクチャ戦略
oidy
1
170
PLoP 2024: The evolution of the microservice architecture pattern language
cer
PRO
0
1.6k
Webの技術スタックで マルチプラットフォームアプリ開発を可能にするElixirDesktopの紹介
thehaigo
2
920
ECS Service Connectのこれまでのアップデートと今後のRoadmapを見てみる
tkikuc
2
210
OpenTelemetryでRailsのパフォーマンス分析を始めてみよう(KoR2024)
ymtdzzz
4
1.6k
リリース8年目のサービスの1800個のERBファイルをViewComponentに移行した方法とその結果
katty0324
5
3.6k
Content Security Policy入門 セキュリティ設定と 違反レポートのはじめ方 / Introduction to Content Security Policy Getting Started with Security Configuration and Violation Reporting
uskey512
1
430
Kotlin2でdataクラスの copyメソッドを禁止する/Data class copy function to have the same visibility as constructor
eichisanden
1
130
Vitest Browser Mode への期待 / Vitest Browser Mode
odanado
PRO
2
1.7k
Amazon Neptuneで始めてみるグラフDB-OpenSearchによるグラフの全文検索-
satoshi256kbyte
4
330
Kubernetes for Data Engineers: Building Scalable, Reliable Data Pipelines
sucitw
1
200
Featured
See All Featured
Faster Mobile Websites
deanohume
304
30k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
27
790
Building Better People: How to give real-time feedback that sticks.
wjessup
363
19k
4 Signs Your Business is Dying
shpigford
180
21k
Git: the NoSQL Database
bkeepers
PRO
425
64k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
664
120k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
The Pragmatic Product Professional
lauravandoore
31
6.3k
Building an army of robots
kneath
302
42k
YesSQL, Process and Tooling at Scale
rocio
167
14k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
7
150
Teambox: Starting and Learning
jrom
132
8.7k
Transcript
You, latency and profiling Filippo Valsorda — @FiloSo/le
What is fast?
A fast hash… … can hash many MB per second
A fast regex… … can process many MB/s
A fast database… … can store many GB per second
A fast database… … can store many GB per second
… or replies to queries in a few ms
A fast website… … can handle many requests … or
loads in a few milliseconds
A fast API… … can handle many clients … or
answers in a few milliseconds
Fast is… Throughput and Latency
CPU profiling -cpuprofile /debug/pprof/profile
SIGPROF CPU profiling
runtime/proc.go
h>p.Handler 1 Running on CPU h>p.Handler 2 I/O I/O h>p.Handler
3 I/O h>p.Handler 4 I/O CPU profiling
Running on CPU I/O I/O I/O I/O h>p.Handler 1 h>p.Handler
2 h>p.Handler 3 h>p.Handler 4 CPU profiling
I/O Observed latency h>p.Handler 1 CPU profiling
func Write(data []byte) { for i := 0; i <
50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } } func Hash(data []byte) { for i := 0; i < 50; i++ { sha256.Sum256(data) } }
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total go tool pprof -web
slowserver cpu.pprof
None
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total $ time
curl http://127.0.0.1:12345/write-no-hash 7.692 total
Latency is not all about CPU
CPU profiling is for throughput The tracer is for latency
$ time curl http://127.0.0.1:12345/write-no-hash 7.692 total go tool trace -pprof=syscall
trace.out > syscall.pprof go tool pprof -web syscall.pprof
None
The tracer Detailed nanosecond-level log of execuOon events: • gorouOne
scheduling, blocking • syscalls, network, I/O • garbage collecOon
•log files are BIG •25% overhead (aWer Go 1.7) •symbols
are embedded (Go 1.7) The tracer
•Full event context •GorouOne metadata and history •Discrete events, not
samples Trace vs. profile
CPU profiling Total: 20 samples
Tracing events
import "runtime/trace" import "net/http/pprof" /debug/pprof/trace?seconds=5 The tracer
None
go tool trace -pprof=TYPE trace.out Supported profile types are: -
net: network blocking profile - sync: synchronization blocking profile - syscall: syscall blocking profile - sched: scheduler latency profile
go tool trace -pprof=syscall trace.out > syscall.pprof go tool pprof
-web syscall.pprof
go tool trace -pprof=syscall trace.out > syscall.pprof go-torch -b syscall.pprof
&& open torch.svg
None
None
-pprof=syscall func Write(data []byte) { for i := 0; i
< 50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } }
-pprof=sync // Block waits on a channel func Block() {
<-time.After(1 * time.Second) }
-pprof=sync
-pprof=sync
-pprof=net func Download() []byte { r, err := http.Get("https://golang.org/doc/go1.8") body,
err := ioutil.ReadAll(r.Body) r.Body.Close() return body }
-pprof=net
-pprof=net
-pprof=net
None
WriIng our own profile prof := make(map[uint64]pprof.Record) for _, ev
:= range events { if ev.Type != trace.EvGoBlockNet { continue } rec := prof[ev.StkID] rec.Stk = ev.Stk rec.N++ rec.Time += ev.Link.Ts - ev.Ts prof[ev.StkID] = rec } pprof.BuildProfile(prof).Write(os.Stdout)
var childG = make(map[uint64]struct{}) var lastGLen int for { for
_, ev := range events { if ev.Type != trace.EvGoCreate { continue } if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) { continue } childG[ev.Args[0]] = struct{}{} } if len(childG) == lastGLen { break } lastGLen = len(childG) }
tracefocus tracefocus -filter=ServeHTTP trace.out > net.pprof github.com/FiloSottile/tracetools
None
None
Build more tools! • Focus on gorouOne number • Aggregate
all blocking types • VisualizaOons • …
tracehist tracehist -filter=Close trace.out github.com/FiloSottile/tracetools
None
Build more tools! Profile latency!
QuesIons? Filippo Valsorda @FiloSo/le hi@filippo.io