Diff'ing Go library API changes

Introducing https://github.com/motemen/gompatible

1. Diff’ing Go library
 API changes Go Conference 2015 Winter
 @motemen, motemen.github.io
2. About me - @motemen on GitHub/Twitter/Hatena - Chief engineer at Hatena - Loves: Go, git - Wrote: ghq, gore and more
3. Diff’ing Go library API changes - Introduction of a tool recently I’ve written: gompat - Motivation, overview & Implementation
4. Motivation
5. “go get -u” and pray - Go packages can be obtained as “latest” - Dependent libraries may introduce breaking changes without your notice - Could use “vendoring” or “revision locking”
6. How can we: - Know dependent libraries’ API changes? - Know to what version to update them? - Know if we are introducing breaking changes
 to our package?
7. API changes overview
8. API entities - Package-level and published: • Functions and methods (func) • Types (type) • Variables (var), Constants (const) - May be changed by the package update
9. Changes: the obvious = Unchanged + Added - Deleted
10. ! Breaking - Named API entities with their interface changed • Users have to modify API usages - … except for the “compatible” case (later)
11. Breaking: struct type example - Struct field deleted - Struct field type changed // before type BreakingT struct { Foo string Bar bool } // after type BreakingT struct { Bar int }
12. Breaking: function example - Parameter types, number changed - Return value types, number changed // before func BreakingF(n, m int) int // after func BreakingF(b bool) (int, error)
13. * Compatible - API signatures changed - Users don’t need their API usages
14. Compatible: function example - Function changed to variadic // before func CompatibleF(n, m int) // after func CompatibleF(n, m int, opts ...string)
15. Compatible: struct type example - Public field added * Breaking for T{x, y, z} initializers // before type CompatibleT struct { Foo int } // after type CompatibleT struct { Foo int Bar bool }
16. Compatible: other cases - Value changed from const to var - Function parameter type became more general • eg. *bytes.Buffer to io.Reader
17. API Changes = Unchanged + Added * Compatible ! Breaking - Deleted
18. Enter Gompatible
19. ! motemen/gompatible - Compares API of a package under two revisions - Produces change kinds per API entities • Added / Removed / Breaking / Compatible
20. cmd/gompat - Main entrypoint of gompatible - Usage: • gompat rev..rev package
21. Inside Gompatible
22. Gompatible tasks - Mostly achieved by std packages: • Load packages at revisions • Load API signatures from packages • Compare type information
23. go/types - Formerly x/tools/go/types - Generates type information from source ASTs - Entrypoint: *types.Package • pkg.Scope().Lookup(“main”) → *types.Func • fun.Type() → *types.Signature • sig.Params(), sig.Results() → *types.Tuple
24. go/types - Can query on types • types.Implements(Type, *Interface) • types.AssignableTo(Type, Type)
25. go/types - It’s a lot of work (and maybe chore) to: • Convert import path to source code directory • Filtering out e.g. *_test.go or *_windows.go’s • Parse source files and combine them as package • Then process typings by go/types
26. x/tools/go/loader - Package path and filenames to *loader.Program • Syntactically parsed and typed packages - Built for command line tools • loader.Config.FromArgs([]string, bool) • Users: x/tools/cmd/… - (Can see loader.FromArgsUsage inside oracle -help)
27. x/tools/go/loader: Usage // set up configuration conf := &loader.Config{ Build: &build.Context{…}, ParserMode: parser.ParseComments, TypeChecker: types.Config{…}, } // specify package path & files conf.CreateFromFilenames(path, files...) // load the program program, _ := conf.Load() // access packages for _, pkg := range program.Created {...}
28. go/build - Used by loader.Config - Handles build information of Go source code files - Know target files under some GOOS, -tag, etc. • Convert import path to source code directory • Filtering out e.g. *_test.go or *_windows.go’s
29. go/build - Entrypoint: *build.Context • Set GOOS or BuildTags and call ImportDir() • Can override ReadDir, OpenFile ‣ Can Build source code on virtual filesystems
30. ! sourcegraph/go-vcs - Virtual source tree at revisions on git/mercurial - Implements x/tools/godoc/vfs.FileSystem • Open(string) (ReadSeekCloser, error) • ReadDir(string) ([]os.FileInfo, error)
31. Inside gompatible - Assign sourcegraph/go-vcs's implementation to go/build.Context - Use go/loader with the build context to obtain type information of two packages - Compare APIs using compatibility rules
32. TODO - Support for: • Vendoring • Mercurial (hg) • More complicated changes - Web interface
33. Diff’ing Go library API changes https://motemen.github.io/