Share
Embed
トルテリリースまでの Go Tips 16/torte-go-tips-16
CA.go #2 発表資料
https://cyberagent.connpass.com/event/64974/
Take
September 06, 2017
Transcript
59 slides
-
τϧςϦϦʔε·Ͱͷ Go Tips 16 த ݑ גࣜձࣾτϧς / αʔόʔαΠυΤϯδχΞ CA.go
#2 2017.09.06 -
• Go on Docker 1. timezone 2. static build •
Upload Images 3. Exif আڈ 4. Orientation 5. ύϑΥʔϚϯε • Terminate 6. Signal ϋϯυϦϯά 7. context ͱ cancel ͱ goroutine GO TIPS 16 • JSON 8. json.Marshaler 9. struct embed 10. omitempty • Credentials 11. ڥม 12. IAMೝূ • Logging 13. zap 14. access log • Test & Error handling 15. gomock 16. errors -
• alpine linux → ca_certificates, tzdataΛผ్install • single binary →
static build → go-bindata Go on Docker -
• time.LoadLocation Go on Docker // LoadLocation returns the Location
with the given name. // // If the name is "" or "UTC", LoadLocation returns UTC. // If the name is "Local", LoadLocation returns Local. // // Otherwise, the name is taken to be a location name corresponding to a file // in the IANA Time Zone database, such as "America/New_York". // // The time zone database needed by LoadLocation may not be // present on all systems, especially non-Unix systems. // LoadLocation looks in the directory or uncompressed zip file // named by the ZONEINFO environment variable, if any, then looks in // known installation locations on Unix systems, // and finally looks in $GOROOT/lib/time/zoneinfo.zip. func LoadLocation(name string) (*Location, error) { -
• alpine Go on Docker FROM alpine:3.6 RUN apk --no-cache
add ca-certificates tzdata -
• static build Go on Docker CGO_ENABLED=0 go build -a
-tags netgo \ -installsuffix netgo -ldflags '-extldflags "-static"' ढจԽʜ -
• Exif ফ͞ͳ͍ͱ… • ػछใ • Ґஔใ • ճససใ •
etc… ͕࿙ΕΔ Upload Images -
• JPEG decode → encode ͢Δ͚ͩͰ Exif औΓআ͚Δ͕… • Orientation
(࣮ࡍͷը૾Λදࣔ͢Δํ) ͜Ε͕ܽଛ͢Δͱճసͨ͠Γసͨ͠ঢ়ଶ Ͱදࣔ͞Εͯ͠·͏ Upload Images -
• Go Ͱ Exif Λ parse ͢Δํ๏ • https://github.com/golang/go/issues/4341 ͔ͳΓલ͔Βٞ͞Ε͍ͯΔ͕ਐḿ͕ͳ͍
• αʔυύʔςΟͷϥΠϒϥϦΛ ͏͔͠ແ͍ (or ࣗ࡞) Upload Images -
Upload Images IUUQXXXDJQBKQTUEEPDVNFOUTK%$+QEG ▪ ը૾ํ Orientation ߦͱྻͷ؍͔Βݟͨɺը૾ͷํɻ Tag = 274
(112.H) Type = SHORT Count = 1 Default = 1 1 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷ্(visual top)ɺ0 ൪ͷྻ͕ࠨଆ(visual left-hand side)ͱͳΔɻ 2 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷ্ɺ0 ൪ͷྻ͕ӈଆ(visual right-hand side)ͱͳΔɻ 3 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷԼ(visual bottom)ɺ0 ൪ͷྻ͕ӈଆͱͳΔɻ 4 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷԼɺ0 ൪ͷྻ͕ࠨଆͱͳΔɻ 5 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷࠨଆɺ0 ൪ͷྻ্͕ͱͳΔɻ 6 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷӈଆɺ0 ൪ͷྻ্͕ͱͳΔɻ 7 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷӈଆɺ0 ൪ͷྻ͕ԼͱͳΔɻ 8 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷࠨଆɺ0 ൪ͷྻ͕ԼͱͳΔɻ ͦͷଞ = ༧ 1ΑΓҾ༻ • Exif ͷن֨ॻΛಡ͏ -
• Exif ͷن֨ॻΛಡ͏ ▪ ը૾ํ Orientation ߦͱྻͷ؍͔Βݟͨɺը૾ͷํɻ Tag = 274
(112.H) Type = SHORT Count = 1 Default = 1 1 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷ্(visual top)ɺ0 ൪ͷྻ͕ࠨଆ(visual left-hand side)ͱͳΔɻ 2 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷ্ɺ0 ൪ͷྻ͕ӈଆ(visual right-hand side)ͱͳΔɻ 3 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷԼ(visual bottom)ɺ0 ൪ͷྻ͕ӈଆͱͳΔɻ 4 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷԼɺ0 ൪ͷྻ͕ࠨଆͱͳΔɻ 5 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷࠨଆɺ0 ൪ͷྻ্͕ͱͳΔɻ 6 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷӈଆɺ0 ൪ͷྻ্͕ͱͳΔɻ 7 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷӈଆɺ0 ൪ͷྻ͕ԼͱͳΔɻ 8 = 0 ൪ͷߦ͕Ͱݟͨͱ͖ͷը૾ͷࠨଆɺ0 ൪ͷྻ͕ԼͱͳΔɻ ͦͷଞ = ༧ Upload Images IUUQXXXDJQBKQTUEEPDVNFOUTK%$+QEG 1ΑΓҾ༻ ຊޠͳͷͰಡΈ͍͢ʂʂ ࣮͍ͨ͠ͱݴͬͯͳ͍ -
• αʔυύʔςΟϥΠϒϥϦɺπʔϧ • ImageMagick • GraphicsMagick • GD Graphics Library
(libgd) • libvips • Pure Go • github.com/BurntSushi/graphics-go • github.com/anthonynsimon/bild • github.com/disintegration/imaging • github.com/disintegration/gift Upload Images -
• ϕϯνϚʔΫ݁Ռ Upload Images BenchmarkGraphicsgo_LargeImage-8 1 4686575195 ns/op BenchmarkBild_LargeImage-8 3
460527153 ns/op BenchmarkImaging_LargeImage-8 10 115713079 ns/op BenchmarkGift_LargeImage-8 5 237302274 ns/op BenchmarkGraphicsgo-8 20 88962194 ns/op BenchmarkBild-8 200 7481979 ns/op BenchmarkImaging-8 500 2507617 ns/op BenchmarkGift-8 300 4335670 ns/op ࣌ܭճΓʹճసͯ͠ࠨӈస͢Δͱ͍͏ॲཧΛɺ Ұ؟ͰࡱӨ͞Εͨେ͖͍ࣸਅ .#͑ ͱɺ খ͍͞ը૾ ,#ఔ ͷ௨ΓͰࢼߦɻ -
• ϕϯνϚʔΫ݁Ռ Upload Images BenchmarkGraphicsgo_LargeImage-8 1 4686575195 ns/op BenchmarkBild_LargeImage-8 3
460527153 ns/op BenchmarkImaging_LargeImage-8 10 115713079 ns/op BenchmarkGift_LargeImage-8 5 237302274 ns/op BenchmarkGraphicsgo-8 20 88962194 ns/op BenchmarkBild-8 200 7481979 ns/op BenchmarkImaging-8 500 2507617 ns/op BenchmarkGift-8 300 4335670 ns/op ࣌ܭճΓʹճసͯ͠ࠨӈస͢Δͱ͍͏ॲཧΛɺ Ұ؟ͰࡱӨ͞Εͨେ͖͍ࣸਅ .#͑ ͱɺ খ͍͞ը૾ ,#ఔ ͷ௨ΓͰࢼߦɻ github.com/disintegration/imaging ࠾༻ -
Upload Images func applyOrientation(img image.Image, meta *exif.Exif) (image.Image, error) {
tag, err := meta.Get(exif.Orientation) if err != nil { return nil, errors.WithStack(err) } orientation, err := tag.Int(0) if err != nil { return nil, errors.WithStack(err) } switch orientation { case 1: return img, nil case 2: img = imaging.FlipH(img) case 3: img = imaging.FlipH(img) img = imaging.FlipV(img) case 4: img = imaging.FlipV(img) case 5: img = imaging.Rotate270(img) img = imaging.FlipH(img) case 6: img = imaging.Rotate270(img) case 7: img = imaging.Rotate90(img) img = imaging.FlipH(img) case 8: img = imaging.Rotate90(img) default: return nil, errors.New("invalid orientation") } } -
• ϓϩηε͍ͭऴྃ͞ΕΔ͔Θ͔Βͳ͍ • API جຊతʹ͍ͭࢮΜͰେৎͳΑ͏ ʹઃܭ͞Ε͍ͯΔ (جຊతʹεςʔτϨεͰτϥϯβΫγϣϯॲཧ͕͋Δ) • Batch ్தͰࢮ͵ͱࠔΔ
Terminate 4JHOBM)BOEMJOH -
• Signal ͕ඈΜͰ͖ͨ࣌ʹऴྃॲཧΛߦ͏ Terminate func terminateHandler(cancel func()) { sigCh :=
make(chan os.Signal) signal.Notify(sigCh, syscall.SIGTERM, syscall.SIGHUP, syscall.SIGINT, syscall.SIGQUIT, ) go func() { sig := <-sigCh log.Printf("Terminated by SIGNAL: %v\n", sig) cancel() }() } ˞syscall.SIGKILLϋϯυϦϯάͰ͖ͳ͍ -
Terminate func main() { ctx, cancel := context.WithCancel(context.Background()) terminateHandler(cancel) }
func exec(ctx context.Context) { ctx, cancel := context.WithTimeout(ctx, execTimeout) defer cancel() ch := make(chan *Record, channelBufferSize) go fetchRecords(ctx, ch) for { select { case <-ctx.Done(): return case record := <-ch: // 1Ϩίʔυͣͭߦ͏ԿΒ͔ͷॲཧ } } } -
Terminate func fetchRecords(ctx context.Context, ch chan<- *Record) (err error) {
defer func() { if cause := recover(); cause != nil { if e, ok := cause.(error); ok { err = e } else { // Τϥʔϩάͱ͔ } } }() // ࣮ࡍʹϨίʔυΛऔͬͯ͘Δॲཧ } -
• json.Marshaler Λ࣮ • MarshalJSON ͷதͰ json.Marshal ΛݺͿ ͱ stack
overflow ͢Δ࣌ͷରॲ๏ JSON IUUQTQMBZHPMBOHPSHQ6X"&-R -
JSON IUUQTQMBZHPMBOHPSHQ6X"&-R • json.Marshaler Λ࣮ͨ͠߹ʹ stack overflow ͢Δ࣌ͷରॲ๏ package
main import ( "encoding/json" "fmt" ) type Obj struct { Text string } func (o *Obj) MarshalJSON() ([]byte, error) { // ԿΒ͔ͷॲཧ type alias Obj return json.Marshal((*alias)(o)) } func main() { obj := &Obj{ Text: "json.Marshaler", } d, _ := json.Marshal(obj) fmt.Println(string(d)) } -
JSON IUUQTQMBZHPMBOHPSHQ6X"&-R • json.Marshaler Λ࣮ͨ͠߹ʹ stack overflow ͢Δ࣌ͷରॲ๏ package
main import ( "encoding/json" "fmt" ) type Obj struct { Text string } func (o *Obj) MarshalJSON() ([]byte, error) { // ԿΒ͔ͷॲཧ type alias Obj return json.Marshal((*alias)(o)) } func main() { obj := &Obj{ Text: "json.Marshaler", } d, _ := json.Marshal(obj) fmt.Println(string(d)) } -
• struct ͷຒΊࠐΈ • struct in struct flat •
interface in struct nest IUUQTQMBZHPMBOHPSHQ[Q'(*XS(X' JSON -
• struct ͷຒΊࠐΈ • struct in struct flat •
interface in struct nest package main import ( "encoding/json" "fmt" ) type Interface interface{} type OuterA struct { Interface Property string } type InnerA struct { InnerProperty string } func main() { objA := &OuterA{ Interface: &InnerA{InnerProperty: "inner"}, Property: "outer", } d, _ := json.Marshal(objA) fmt.Println(string(d)) } IUUQTQMBZHPMBOHPSHQ[Q'(*XS(X' JSON -
• struct in struct flatten ͞ΕΔ • interface in
struct nest ͞ΕΔ • ͑͑ͬʂʁ package main type Interface interface{} type OuterA struct { Interface Property string } type InnerA struct { InnerProperty string } func main() { objA := &OuterA{ Interface: &InnerA{InnerProperty: "inner"}, Property: "outer", } d, _ := json.Marshal(objA) fmt.Println(string(d)) } IUUQTQMBZHPMBOHPSHQ[Q'(*XS(X' { "Interface": { "InnerProperty": "inner" }, "Property": "outer" } JSON ͑͑ͬʂʁ -
• time.Time omitempty Ͱ͖ͳ͍ • https://github.com/golang/go/issues/4357 • https://github.com/golang/go/issues/11939 •
ͭΒ͍ • ྑ͘ͳ͍͚Ͳ *time.Time ͏͔͠ແͦ͞͏ JSON -
• ೝূใΛͲ͜ʹ͔࣋ͭ • ॳ: ڥม → TaskDefinition, Terraform ʹͨॻ͖ →
ͪΖΜฏจͰ Credentials -
• EC2 System Manager Parameter Store → AWS ͳΒ͜ΕҰʁ →
KMS ͷ伴Ͱ҉߸Խͯ͘͠ΕΔ • ଞͳΒ etcd, consul ͱ͍͏બࢶ • github.com/vrischmann/envconfig Λ wrap ͯ͠ೝূใ͚ͩ Parameter Store ࢀর Credentials -
• ೝূϦΫΤετ • Amazon Elasticsearch Service ͳͲΛ͏࣌ʹ Signing AWS API
Request ʹ ै͏ඞཁ͕͋Δ • ରԠ͍ͯ͠ͳ͍αʔυύʔςΟͷΫϥΠΞϯτ Λͦͷ··͏ͳΒ http.RoundTripper Λ ࣮͢Δ Credentials -
Credentials // Transport implement http.RoundTripper. type Transport struct {
Base http.RoundTripper } func (t *Transport) base() http.RoundTripper { if t.Base != nil { return t.Base } return http.DefaultTransport } // RoundTrip wraps http.DefaultTransport.RoundTrip for AWS signed request. func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error) { // clone request body var body io.ReadSeeker buff := &bytes.Buffer{} if req.Body != nil { buff.ReadFrom(req.Body) req.Body = ioutil.NopCloser(buff) body = bytes.NewReader(buff.Bytes()) } signer := v4.NewSigner(credentials.AWSSession.Config.Credentials) _, err := signer.Sign(req, body, awsServiceName, awsRegion, time.Now()) if err != nil { return nil, err } return t.base().RoundTrip(req) } -
• zap • JSON ܗࣜͷϩά • ͍Ζ͍ΖͭΒ͔ͬͨ json package Λ͏
ඞཁ͕ͳ͍ • ύϑΥʔϚϯεʹ༏Ε͍ͯΔ Logging HJUIVCDPNVCFSHP[BQ -
Logging logger, _ := zap.NewProduction() defer logger.Sync() logger.Info("failed to
fetch URL", zap.String("url", url), zap.Int("attempt", 3), zap.Duration("backoff", time.Second), ) -
• access/app log • RequestID Λੜͯ͠༩͢Δ • ϨεϙϯεͰ X-Request-ID ϔομΛฦ͢
• 1ͭͷϦΫΤετͷྲྀΕ͕͑ΔͷͰɺ σόοάָ͕ʹʂ Logging -
• gomock • interface Λݩʹ mock Λੜ • ಛʹ࣍ͷΑ͏ͳ֎෦ͷঢ়ଶʹґଘ͢ΔͷΛ mock
Խ͢Δͱศར • σʔλετΞपΓ • ֎෦ API पΓ • ཚͳͲ • ςετͰ༗༻ Test & Error handling HJUIVCDPNHPMBOHNPDL -
• gomock • interface Λݩʹ mock Λੜ • ಛʹ࣍ͷΑ͏ͳ֎෦ͷঢ়ଶʹґଘ͢ΔͷΛ mock
Խ͢Δͱศར • σʔλετΞपΓ • ֎෦ API पΓ • ཚͳͲ • ςετͰ༗༻ Test & Error handling HJUIVCDPNHPMBOHNPDL package s3 import "io" type Client interface { Upload(bucket string, key string, file io.ReadSeeker) error } type implement struct{} func (s *implement) Upload(bucket string, key string, file io.ReadSeeker) error { // implement return nil } func New() Client { return &implement{} } -
• gomock • interface Λݩʹ mock Λੜ • ಛʹ࣍ͷΑ͏ͳ֎෦ͷঢ়ଶʹґଘ͢ΔͷΛ mock
Խ͢Δͱศར • σʔλετΞपΓ • ֎෦ API पΓ • ཚͳͲ • ςετͰ༗༻ Test & Error handling HJUIVCDPNHPMBOHNPDL package main import // தུ const bucketName = "{bucket_name}" var s3Client = s3.New() func UploadImage(fileName string, file io.ReadSeeker) error { fileHeader := make([]byte, 512) file.Read(fileHeader) mimeType := http.DetectContentType(fileHeader) _, err := file.Seek(0, io.SeekStart) if err != nil { return err } var ext string switch mimeType { case "image/png": ext = ".png" case "image/jpeg": ext = ".jpg" default: return errors.New("unsupported file type") } return s3Client.Upload(bucketName, fileName+ext, file) } -
• gomock • interface Λݩʹ mock Λੜ • ಛʹ࣍ͷΑ͏ͳ֎෦ͷঢ়ଶʹґଘ͢ΔͷΛ mock
Խ͢Δͱศར • σʔλετΞपΓ • ֎෦ API पΓ • ཚͳͲ • ςετͰ༗༻ Test & Error handling HJUIVCDPNHPMBOHNPDL package main import ( "os" "path/filepath" "testing" // தུ "github.com/golang/mock/gomock" ) func TestUploadImage(t *testing.T) { ctrl := gomock.NewController(t) defer ctrl.Finish() mockS3Client := s3.NewMockClient(ctrl) mockS3Client.EXPECT().Upload( bucketName, "test.jpg", gomock.Any(), ).Return(nil) s3Client = mockS3Client filePath, _ := filepath.Abs("testdata/test.jpg") file, _ := os.Open(filePath) err := UploadImage("test", file) if err != nil { t.Error(err) } } -
• Error handling • ඪ४ͷ errors package ͷΘΓʹ͏ • ಛʹ
WithStack ͱ͍͏ϝιου͕ศར • ඪ४ͷ error Λड͚औΔՕॴͰ WithStack Ͱ stack trace Λ༩͢Δ͜ͱͰϩά͔ΒΤϥʔ ൃੜՕॴ͕͔ΓσόοάḿΔ Test & Error handling HJUIVCDPNQLHFSSPST -
• Error handling • ඪ४ͷ errors package ͷΘΓʹ͏ • ಛʹ
WithStack ͱ͍͏ϝιου͕ศར • ඪ४ͷ error Λड͚औΔՕॴͰ WithStack Ͱ stack trace Λ༩͢Δ͜ͱͰϩά͔ΒΤϥʔ ൃੜՕॴ͕͔ΓσόοάḿΔ Test & Error handling HJUIVCDPNQLHFSSPST package main import ( "strconv" "github.com/pkg/errors" ) func main() { } func errorMethod() error { i, err := strconv.Atoi("") if err != nil { return errors.WithStack(err) } return nil } -
• Error handling • ඪ४ͷ errors package ͷΘΓʹ͏ • ಛʹ
WithStack ͱ͍͏ϝιου͕ศར • ඪ४ͷ error Λड͚औΔՕॴͰ WithStack Ͱ stack trace Λ༩͢Δ͜ͱͰϩά͔ΒΤϥʔ ൃੜՕॴ͕͔ΓσόοάḿΔ Test & Error handling HJUIVCDPNQLHFSSPST package main import ( "strconv" "github.com/pkg/errors" ) func main() { } func errorMethod() error { i, err := strconv.Atoi("") if err != nil { return errors.WithStack(err) } return nil } TUSDPOW"UPJQBSTJOHJOWBMJETZOUBY NBJOFSSPS.FUIPE 6TFSTXXHPTSDFSSPSTNBJOHP NBJONBJO 6TFSTXXHPTSDFSSPSTNBJOHP SVOUJNFNBJO VTSMPDBM$FMMBSHPMJCFYFDTSDSVOUJNFQSPDHP SVOUJNFHPFYJU VTSMPDBM$FMMBSHPMJCFYFDTSDSVOUJNFBTN@BNET -
͜͜·Ͱొ͠·ͤΜͰ͕ͨ͠ɺ • ϑϨʔϜϫʔΫ • echo (github.com/labstack/echo) • package ཧ •
glide (github.com/Masterminds/glide) • ORM • xorm (github.com/go-xorm/xorm) ͬͯ·͢ɻ ·ͱΊ -
• εϥΠυதͰհͨ͠ package Ұཡ • github.com/disintegration/imaging • github.com/vrischmann/envconfig • github.com/uber-go/zap
• github.com/golang/mock • github.com/pkg/errors ·ͱΊ