Simulating a real-world system in Go

Go's concurrency model makes it easy to develop scalable servers and data pipelines.
Many of the patterns we use in developing concurrent code mirror structures in real-world systems.
In this talk, I'll present a simulation of a small real world system and show how variations in the design impact the system's performance.

Presented at dotGo in Paris on November 6, 2017

Sourcegraph write-up, includes much of the script: https://about.sourcegraph.com/go/simulating-a-real-world-system-in-go/

Transcript

1. Simulating a real-world system in Go Sameer Ajmani Go team lead, Google dotGo, Paris November 2017
2. By Sajibabu79 (Own work) [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
3. By Mtattrain (Own work) [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0)], via Wikimedia Commons
4. By Eden, Janine and Jim from New York City (Voting Line IX) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons
5. https://www.goodfreephotos.com/people/people-in-a-coffee-shop.jpg.php
6. The simulator Generate customers Collect latency distribution Measure throughput and utilization Order coffee & wait Order coffee & wait Order coffee & wait Order coffee & wait Order coffee & wait Order coffee & wait Customers Wait times
7. An ideal latte Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms Time
8. The ideal function Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms func idealBrew() latte { grounds := grindCoffee(grinder) coffee := makeEspresso(espressoMachine, grounds) milk := steamMilk(steamer) return makeLatte(coffee, milk) } Time
9. Ideal performance Ideal Ideal Better Better
10. Reality strikes func grindCoffee(grinder *machine) grounds { grinder.add(runPhase(*grindTime)) return grounds(0) } func makeEspresso(espressoMachine *machine, grounds grounds) coffee { espressoMachine.add(runPhase(*pressTime)) return coffee(grounds) } func steamMilk(steamer *machine) milk { steamer.add(runPhase(*steamTime)) return milk(0) } By Tijuana Brass at en.wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], from Wikimedia Commons
11. Lesson: Use the race detector ================== WARNING: DATA RACE Read at 0x00c42009c328 by goroutine 9: main.(*sampler).add() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/perf.go:53 +0x72 main.grindCoffee() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:151 +0x99 main.idealBrew() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:130 +0x4e main.main.func2() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:596 +0x3d main.perfTest.func2() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/perf.go:182 +0xe4 Previous write at 0x00c42009c328 by goroutine 8: main.(*sampler).add() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/perf.go:57 +0xba main.grindCoffee() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:151 +0x99 main.idealBrew() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:130 +0x4e main.main.func2() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:596 +0x3d main.perfTest.func2() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/perf.go:182 +0xe4
12. var kitchen sync.Mutex func lockingBrew() latte { kitchen.Lock() defer kitchen.Unlock() grounds := grindCoffee(grinder) coffee := makeEspresso(espressoMachine, grounds) milk := steamMilk(steamer) return makeLatte(coffee, milk) } By David Adam Kess (Own work) [CC BY-SA 4.0], via Wikimedia Commons
13. Locking performance Ideal (uncontended) Ideal (uncontended) Whole-kitchen locking Whole-kitchen locking
14. Fine-grain locking func lockingGrind() grounds { grinder.Lock() defer grinder.Unlock() return grindCoffee(grinder) } func lockingPress(grounds grounds) coffee { espressoMachine.Lock() defer espressoMachine.Unlock() return makeEspresso(espressoMachine, grounds) } func lockingSteam() milk { steamer.Lock() defer steamer.Unlock() return steamMilk(steamer) } By Tijuana Brass at en.wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], from Wikimedia Commons
15. Fine-grain locking performance Fine-grain locking Fine-grain locking Ideal (uncontended) Ideal (uncontended) W hole-kitchen locking Whole-kitchen locking
16. Lesson: Minimize locked duration Fine-grain locking Fine-grain locking Ideal (uncontended) Ideal (uncontended) W hole-kitchen locking Whole-kitchen locking
17. Structural limits Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms CPU1 Time Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms Grind beans 1ms Make espresso 1ms Steam milk 1ms CPU2 CPU3 CPU4 CPU5 CPU6 Grind beans 1ms Make espresso 1ms Grind beans 1ms
18. By dumbonyc (Brooklyn Ice Cream Factory) [CC BY-SA 2.0 (https://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons
19. Two sets of machines func multiGrind() grounds { m := <-grinders grounds := grindCoffee(m) grinders <- m return grounds } func multiPress(grounds grounds) coffee { m := <-espressoMachines coffee := makeEspresso(m, grounds) espressoMachines <- m return coffee } func multiSteam() milk { m := <-steamers milk := steamMilk(m) steamers <- m return milk } By Tijuana Brass at en.wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], from Wikimedia Commons
20. Multi machine performance Ideal & Multiple machines Fine-grain locking Ideal & Multiple machines Fine-grain locking W hole-kitchen locking Whole-kitchen locking
21. Lesson: Identify the limiting factor
22. \ By HAO XING (BIZ INDIA-STARBUCKS-BIZPLUS 2 SE) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons
23. A coffee pipeline func (p *linearPipeline) grinder() { for o := range p.orders { o.grounds = grindCoffee(p.grinderMachine) p.ordersWithGrounds <- o } close(p.ordersWithGrounds) } func (p *linearPipeline) presser() { for o := range p.ordersWithGrounds { o.coffee = makeEspresso( p.espressoMachine, o.grounds) p.ordersWithCoffee <- o } close(p.ordersWithCoffee) } func (p *linearPipeline) steamer() { for o := range p.ordersWithCoffee { o.milk <- steamMilk(p.steamerMachine) } close(p.done) } By Tijuana Brass at en.wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], from Wikimedia Commons
24. Pipeline vs. locking Unbuffered pipeline Fine-grain locking & Unbuffered pipeline Fine-grain locking Ideal (uncontended) Ideal (uncontended) W hole-kitchen locking Whole-kitchen locking
25. Buffered vs. unbuffered Buffered pipelines Buffered pipelines Unbuffered pipeline Unbuffered pipeline Ideal (uncontended) Ideal (uncontended) W hole-kitchen locking Whole-kitchen locking
26. Major vs. minor optimizations
27. Many more scenarios
28. Remove structural barriers to parallelism Study real-world systems for inspiration
29. Get the simulator More simulation modes! More controls! More data! • Americano (no “steam” stage) & Espresso (no “steam” or “make” stage) • Steam milk in parallel with grinding beans & making espresso • Controls: stage durations, jitter, bounded request queues, arrival rate • Data: latency distributions, CPU utilization, queue drop rate go get github.com/Sajmani/dotgo/coffee