Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Go Secrets & Gotchas / Yarel Maman

Go Secrets & Gotchas / Yarel Maman

This talk will try to give you a taste of various secrets and gotchas in the Go language, followed by examples.

At first, Maybe you heard about the language or perhaps you’re starting a new job writing Go. So you go through the wonderful “Tour of Go”, then you go over “Effective Go”, maybe watch a few fun videos of Rob Pike on YouTube. Write a small CLI app or a web service. And then you might think you have a notion of Go as a simple language and that it might be difficult to surprise you. Well - You might be surprised!

Over the time, perhaps through a bug or a comment in a Code review, you discover small (and sometimes, significant) details about the language, about its standard libraries, and about its runtime behavior, that keeps you intrigued, and letting you know you still have a lot to learn.

This talk will present a few common gotchas in Go. Some of these I often see in code reviews with Go developers. It’s less about style and more about “hidden” behavior you may want to be aware of.

GopherCon Israel

February 11, 2019
Tweet

More Decks by GopherCon Israel

Other Decks in Programming

Transcript

  1. Passing a slice Language func appendToSlice(someSlice []int) { someSlice =

    append(someSlice, 2) } func main() { someSlice := make([]int, 0) fmt.Printf("Slice before %v\n", someSlice) appendToSlice(someSlice) fmt.Printf("Slice after %v\n", someSlice) }
  2. Passing a slice Language func appendToSlice(someSlice []int) { someSlice =

    append(someSlice, 2) } func main() { someSlice := make([]int, 0) fmt.Printf("Slice before %v\n", someSlice) appendToSlice(someSlice) fmt.Printf("Slice after %v\n", someSlice) } Slice before [] Slice after [] Output:
  3. Passing a map func setInMap(someMap map[int]int) { someMap[0] = 0

    } func main() { someMap := make(map[int]int) fmt.Printf("Map before %v\n", someMap) setInMap(someMap) fmt.Printf("Map after %v\n", someMap) } Language
  4. Passing a map func setInMap(someMap map[int]int) { someMap[0] = 0

    } func main() { someMap := make(map[int]int) fmt.Printf("Map before %v\n", someMap) setInMap(someMap) fmt.Printf("Map after %v\n", someMap) } Map before map[] Map after map[0:0] Output: Language
  5. • A slice is a struct Slices in a nutshell

    Taken from https://blog.golang.org/go-slices-usage-and-internals (runtime/slice.go) type slice struct { array unsafe.Pointer len int cap int } Language
  6. Maps in a nutshell • A map is a pointer

    to a struct (hmap) (runtime/map.go) func makemap(t *maptype, hint int, h *hmap) *hmap { .. } Language
  7. Going back to our example • appendToSlice - gets a

    copy of a struct func appendToSlice(someSlice []int) { someSlice = append(someSlice, 2) } Language
  8. Going back to our example • appendToSlice - gets a

    copy of a struct func appendToSlice(someSlice []int) { someSlice = append(someSlice, 2) } • setInMap - gets a copy of a pointer (to an hmap struct) func setInMap(someMap map[int]int) { someMap[0] = 0 } Language
  9. m := make(map[int]item, 1) … m[0].id = 1 Compile error:

    cannot assign to struct field m[0].id in map type item struct { id int }
  10. • Map values are not addressable m := make(map[int]item, 1)

    … m[0].id = 1 Compile error: cannot assign to struct field m[0].id in map type item struct { id int }
  11. • Slice values are addressable s := make([]item, 1) …

    s[0].id = 1 • Map values are not addressable m := make(map[int]item, 1) … m[0].id = 1 Compile error: cannot assign to struct field m[0].id in map type item struct { id int }
  12. • Slice values are addressable s := make([]item, 1) …

    s[0].id = 1 • Map values are not addressable m := make(map[int]item, 1) … m[0].id = 1 Compile error: cannot assign to struct field m[0].id in map type item struct { id int }
  13. Slicing a slice “Slicing does not copy the slice’s data.

    It creates a new slice that points to the original array” (Slice internals) s := make([]byte, 5) s := s[2:4] Language
  14. “memory leaks” when slicing var digitRegexp = regexp.MustCompile("[0-9]+") func FindDigits(filename

    string) []byte { b, _ := ioutil.ReadFile(filename) return digitRegexp.Find(b) } Example taken from https://blog.golang.org/go-slices-usage-and-internals Language
  15. “memory leaks” when slicing var digitRegexp = regexp.MustCompile("[0-9]+") func FindDigits(filename

    string) []byte { b, _ := ioutil.ReadFile(filename) return digitRegexp.Find(b) } Example taken from https://blog.golang.org/go-slices-usage-and-internals Example file contents: 00aaaaaaaaaaaa …… (File size - 3.7MB) Language
  16. “memory leaks” when slicing var digitRegexp = regexp.MustCompile("[0-9]+") func FindDigits(filename

    string) []byte { b, _ := ioutil.ReadFile(filename) return digitRegexp.Find(b) } Example taken from https://blog.golang.org/go-slices-usage-and-internals Reads an entire file contents to a byte slice Data = 0xc000090120 Len = 3756660 Cap = 3757172 Slice header of b: Example file contents: 00aaaaaaaaaaaa …… (File size - 3.7MB) Language
  17. “memory leaks” when slicing var digitRegexp = regexp.MustCompile("[0-9]+") func FindDigits(filename

    string) []byte { b, _ := ioutil.ReadFile(filename) return digitRegexp.Find(b) } Example taken from https://blog.golang.org/go-slices-usage-and-internals Reads an entire file contents to a byte slice Data = 0xc000090120 Len = 3756660 Cap = 3757172 Slice header of b: Returns the first group of the regex match by slicing b. Data = 0xc000090120 Len = 2 Cap = 3757172 Returned slice header: Example file contents: 00aaaaaaaaaaaa …… (File size - 3.7MB) Language
  18. “memory leaks” when slicing var digitRegexp = regexp.MustCompile("[0-9]+") func FindDigits(filename

    string) []byte { b, _ := ioutil.ReadFile(filename) return digitRegexp.Find(b) } Example taken from https://blog.golang.org/go-slices-usage-and-internals Possible solution: Reads an entire file contents to a byte slice Data = 0xc000090120 Len = 3756660 Cap = 3757172 Slice header of b: Returns the first group of the regex match by slicing b. Data = 0xc000090120 Len = 2 Cap = 3757172 Returned slice header: Example file contents: 00aaaaaaaaaaaa …… (File size - 3.7MB) Language
  19. “memory leaks” when slicing var digitRegexp = regexp.MustCompile("[0-9]+") func FindDigits(filename

    string) []byte { b, _ := ioutil.ReadFile(filename) return digitRegexp.Find(b) } Example taken from https://blog.golang.org/go-slices-usage-and-internals func CopyDigits(filename string) []byte { b, _ := ioutil.ReadFile(filename) b = digitRegexp.Find(b) c := make([]byte, len(b)) copy(c, b) return c } Possible solution: Reads an entire file contents to a byte slice Data = 0xc000090120 Len = 3756660 Cap = 3757172 Slice header of b: Returns the first group of the regex match by slicing b. Data = 0xc000090120 Len = 2 Cap = 3757172 Returned slice header: Example file contents: 00aaaaaaaaaaaa …… (File size - 3.7MB) Language
  20. “memory leaks” when slicing var digitRegexp = regexp.MustCompile("[0-9]+") func FindDigits(filename

    string) []byte { b, _ := ioutil.ReadFile(filename) return digitRegexp.Find(b) } Example taken from https://blog.golang.org/go-slices-usage-and-internals func CopyDigits(filename string) []byte { b, _ := ioutil.ReadFile(filename) b = digitRegexp.Find(b) c := make([]byte, len(b)) copy(c, b) return c } Possible solution: Reads an entire file contents to a byte slice Data = 0xc000090120 Len = 3756660 Cap = 3757172 Slice header of b: Returns the first group of the regex match by slicing b. Data = 0xc000090120 Len = 2 Cap = 3757172 Returned slice header: Making a copy of the sliced buffer, and returning it. Data = 0xc0004420b0 Len = 2 Cap = 2 Returned Slice header: Example file contents: 00aaaaaaaaaaaa …… (File size - 3.7MB) Language
  21. Interfaces and nil values var counter *int fmt.Println(counter == nil)

    var metric interface{} fmt.Println(metric == nil) metric = counter fmt.Println(metric == nil) Language
  22. Interfaces and nil values var counter *int fmt.Println(counter == nil)

    var metric interface{} fmt.Println(metric == nil) metric = counter fmt.Println(metric == nil) Output: Language
  23. Interfaces and nil values var counter *int fmt.Println(counter == nil)

    var metric interface{} fmt.Println(metric == nil) metric = counter fmt.Println(metric == nil) Output: true Language
  24. Interfaces and nil values var counter *int fmt.Println(counter == nil)

    var metric interface{} fmt.Println(metric == nil) metric = counter fmt.Println(metric == nil) Output: true true Language
  25. Interfaces and nil values var counter *int fmt.Println(counter == nil)

    var metric interface{} fmt.Println(metric == nil) metric = counter fmt.Println(metric == nil) Output: true true false Language
  26. { Go interface values in a nutshell Interface value Dynamic

    Type Type info of T. T implements the interface. Dynamic Value Copy of a concrete value of type T. Language
  27. Interfaces and nil values Language var counter *int var metric

    interface{} fmt.Printf("metric before: V=%v T=%T\n", metric, metric) metric = counter fmt.Printf("metric after: V=%v T=%T\n", metric, metric)
  28. Interfaces and nil values metric before: V=<nil> T=<nil> metric after:

    V=<nil> T=*int Output: Language var counter *int var metric interface{} fmt.Printf("metric before: V=%v T=%T\n", metric, metric) metric = counter fmt.Printf("metric after: V=%v T=%T\n", metric, metric)
  29. Interfaces and nil values func returnsError() error { var p

    *MyError = nil if bad() { p = ErrBad } return p // Will always return a non-nil error. } Example taken from https://golang.org/doc/faq#nil_error Language
  30. Interfaces and nil values • A possible solution func returnsError()

    error { var err error if bad() { err = ErrBad } return err } Language
  31. Interfaces and nil values • A possible solution func returnsError()

    error { var err error if bad() { err = ErrBad } return err } For more, watch “Understanding nil” Language
  32. Using goroutines on loop iterator variables Language values := []int{1,

    2, 3, 4} for _, val := range values { go func() { fmt.Printf("val %v, addr %v\n", val, &val) }() }
  33. Using goroutines on loop iterator variables val 4, addr 0xc000014078

    val 4, addr 0xc000014078 val 4, addr 0xc000014078 val 4, addr 0xc000014078 Output: Language values := []int{1, 2, 3, 4} for _, val := range values { go func() { fmt.Printf("val %v, addr %v\n", val, &val) }() }
  34. Using goroutines on loop iterator variables Language values := []int{1,

    2, 3, 4} for _, val := range values { go func(val int) { fmt.Printf("val %v, addr %v\n", val, &val) }(val) }
  35. Using goroutines on loop iterator variables val 1, addr 0xc000014088

    val 3, addr 0xc00009e000 val 2, addr 0xc000086000 val 4, addr 0xc000014078 Output: Language values := []int{1, 2, 3, 4} for _, val := range values { go func(val int) { fmt.Printf("val %v, addr %v\n", val, &val) }(val) }
  36. Odd solution values := []int{1, 2, 3, 4} for _,

    val := range values { val := val go func() { fmt.Println(val) }() } Credit: Ewa Gillen Language
  37. Odd solution values := []int{1, 2, 3, 4} for _,

    val := range values { val := val go func() { fmt.Println(val) }() } Credit: Ewa Gillen Language
  38. Using goroutines on loop iterator variables Language $ go vet

    -rangeloops goroutine.go ./goroutine.go:14: loop variable val captured by func literal
  39. Defer in a loop for fileName := range fileNameC {

    f, err := os.Open(fileName) if err != nil { /* ERROR HANDLING */ } defer f.Close() … } Language
  40. Copying sync primitives sync/mutex.go // A Mutex is a mutual

    exclusion lock. // The zero value for a Mutex is an unlocked mutex. // A Mutex must not be copied after first use. type Mutex struct { state int32 sema uint32 } • Go sync primitives must not be copied Standard libs
  41. Copying sync primitives sync/mutex.go // A Mutex is a mutual

    exclusion lock. // The zero value for a Mutex is an unlocked mutex. // A Mutex must not be copied after first use. type Mutex struct { state int32 sema uint32 } • Go sync primitives must not be copied // This type must not be copied type Value struct { sync.RWMutex … } • Types containing sync primitives values must not be copied Standard libs
  42. Copying sync primitives type pond struct { sync.Mutex ducks map[string]duck

    } func (p pond) addDuck(duck duck) { p.Lock() p.ducks[duck.name] = duck p.Unlock() } Standard libs
  43. Copying sync primitives type pond struct { sync.Mutex ducks map[string]duck

    } func (p pond) addDuck(duck duck) { p.Lock() p.ducks[duck.name] = duck p.Unlock() } Value receiver Standard libs
  44. Copying sync primitives type pond struct { sync.Mutex ducks map[string]duck

    } func (p pond) addDuck(duck duck) { p.Lock() p.ducks[duck.name] = duck p.Unlock() } Value receiver See “CodeReviewComments” For guidelines Standard libs
  45. Copying sync primitives Use “go vet” to easily detect it

    $ go vet -copylocks mutex.go mutex.go:24: addDuck passes lock by value: sync_copies.pond Standard libs
  46. Using maps concurrently go func() { fmt.Println(m[0]) // DATA RACE!

    }() m[0] = 0 // DATA RACE! • Go’s builtin maps are not safe for concurrent write, or concurrent read and write Standard libs
  47. Using maps concurrently go func() { fmt.Println(m[0]) // DATA RACE!

    }() m[0] = 0 // DATA RACE! • Go’s builtin maps are not safe for concurrent write, or concurrent read and write m[0] = 0 … go fmt.Println(m[0]) go fmt.Println(m[0]) • concurrent read (only) is safe Standard libs
  48. The runtime may panic! fatal error: concurrent map writes goroutine

    1194 [running]: runtime.throw(0x1fe22bf, 0x15) /usr/local/go/src/runtime/panic.go:596 +0x95 fp=0xc420ca8dc8 sp=0xc420ca8da8 runtime.mapassign(0x1e03b40, 0xc420bbb890, 0xc420ca8ea0, 0x1b) /usr/local/go/src/runtime/hashmap.go:499 +0x667 fp=0xc420ca8e68 sp=0xc420ca8dc8 Standard libs
  49. Go Data Race Detector ================== WARNING: DATA RACE Read at

    0x00c0000ac048 by goroutine 6: main.main.func1() gosecrets/maps/example.go:9 +0x64 Previous write at 0x00c0000ac048 by main goroutine: main.main() gosecrets/maps/example.go:12 +0x91 Goroutine 6 (running) created at: main.main() gosecrets/maps/example.go:8 +0x59 ================== go func() { fmt.Println(m[0]) }() m[0] = 0 $ go run -race example.go Standard libs
  50. Community Closing words •Keeping up to date with the language

    development •Reading Go Github issues •Reading the change log of Go releases •Subscribing to newsletters such as: Golang Weekly •Golang-nuts (Google groups) •r/golang (Reddit)