3 secrets to build web APIs in Go

Based on his experience with Splice, Matt Aimonetti explores 3 critical elements that made a big difference when building a startup on top of Go.
Talk given at Gopher SummerFest and Google I/O

1. 3 secrets to build web APIs in Go Matt Aimonetti - splice.com - @mattetti
2. Matt Aimonetti splice: Go, Ruby, JS, Obj-C, C#, C++, C golangbootcamp.com
3. Splice
4. Splice
8. Architecture clients APIs / core logic web rendering
9. Architecture Web API: Synchronization File upload File download Notifications Obj-C C# C Go
10. Architecture file parsing/analysis (sessions, media) deduplication/storage intermediate representation
11. Architecture internal/private API (application logic) “public” API (states retrieval/change)
12. Architecture HTTP Message Bus Long running async tasks
13. Go vs Ruby 10-20% slower to write 20-100x faster to run 60-80% less maintenance required batteries included
14. • Deployment • Routing • Monitoring
16. Deployment Zero downtime deployments bitbucket.org/splice/go.grace careful with goroutines
17. Deployment Version Control CI Storage
18. Deployment
19. Deployment 2014-06-23-183000/api 2014-06-23-204500/api current/api symlink
20. Deployment 2014-06-23-183000/api 2014-06-23-204500/api current/api $ kill -USR2 {api PID} symlink
22. Routing var V42Routes = Endpoints{ // create Ableton Live session { Verb: "POST", Path: "/v42/live/sessions", Handler: handlers.AlsUpload, RequiredParams: handlers.AlsUploadReqParams, Auth: splice.ClientAuth, }, // create Logic session { Verb: "POST", Path: "/v42/logic/sessions", Handler: handlers.LogicUpload, RequiredParams: handlers.LogicUploadReqParams, Auth: splice.ClientAuth, }, }
23. Routing package router type Endpoint struct { Verb string Path string RequiredParams []string Auth func(*splice.ReqEnv, bool) bool Handler splice.Handler }
24. Routing type Endpoints []Endpoint // Activate all the endpoint API routes. func (es Endpoints) Activate(p *pat.PatternServeMux) { for _, e := range es { // [..] set authentication calls // [..] check param requirements handler := splice.ReqWrap(e.Handler, e.Auth) p.Add(e.Verb, e.Path, handler) // [..] collect all CORS verbs accepted for this endpoint so we can create OPTIONS routes. } }
25. Routing func ReqWrap(handler *Handler, checkAuthFn func(*ReqEnv, bool) bool) http.Handler { ! return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { start := time.Now() defer reportRequest(req, start, handler.Metrics) reqUuid := uuid.GenUUID() req := &ReqEnv{ ReqUuid: reqUuid, Request: r, Response: &statusResponseWriter{w, 0}, } // [..] set headers (content type, req ID, build) if !checkAuthFn(req, checkAuth) { return } ! ! !
26. Routing ! ! ! // We must return a 400 and stop here if there was a problem // parsing the request. err := req.ParseParams() if err != nil { // […]return 400 return } // convert a panic into a 500 defer req.handlePanic() handler.Handle(req) }) }
28. Monitoring Monitor all the things! runtime.ReadMemStats http://golang.org/pkg/runtime/#MemStats
29. Lessons learned low memory usage
30. Lessons learned detect issues
31. Lessons learned detect issues
32. Low memory usage use streaming io.Reader/io.Writer io.Copy Encoder/Decoder instead of ReadAll and Unmarshal
33. Monitoring Monitor all the things! Per endpoint: • # of requests • Status codes • Response times
35. @mattetti http://matt.aimonetti.net