Upgrade to Pro — share decks privately, control downloads, hide ads and more …

堅牢なTCPサーバを作るために - katsubushiの知見から/kamakura.go#5

堅牢なTCPサーバを作るために - katsubushiの知見から/kamakura.go#5

kamakura.go #5

FUJIWARA Shunichiro

June 22, 2019
Tweet

More Decks by FUJIWARA Shunichiro

Other Decks in Technology

Transcript

  1. katsubushi - ෼ࢄϢχʔΫID࠾൪ػ • ϢχʔΫͳ࣌ܥྻॱͷInt64 IDΛൃߦ͢Δ • ෼ࢄ؀ڥ(ෳ਺୆)ͰඃΒͣಈ͘ • Memcached

    ProtocolΛ࣮૷͍ͯ͠Δ • Go੡ͷಠ࣮ࣗ૷ϛυϧ΢ΣΞ github.com/kayac/go-katsubushi
  2. ݎ࿚ͳαʔόʁ ݎ࿚ is ... • ϦιʔεϦʔΫ͠ͳ͍ • མͪͳ͍ • (ϓϩμΫτͷੑ্࣭)

    ID͕ܾͯ͠ඃΒͳ͍ • →γϟʔσΟϯά͞ΕͨผDBʹಉҰID͕ൃߦ͞ΕͨΒࢮ
  3. memcached protocol ͷ࠾༻ཧ༝ • ܰྔ • HTTPͷΑ͏ʹ༨ܭͳϔομ͕ͳ͍ • Int64 Λ1ݸฦ͢ͷʹϔομ͕Կඦbyteͱ͔ͪΐͬͱ…

    • ΫϥΠΞϯτ͕ߴ଎ • ౰࣌ओʹ࢖༻͍ͯͨ͠Perlʹ͸Cache::Memcached::Fastͱ ͍͏ߴ଎ͳ࣮૷ ID͸େྔʹൃߦ͢Δ͜ͱ͕͋ΔͷͰ͍ܰͷ͕Α͍
  4. Go Ͱ TCP αʔό net package Ͱ؆୯ʹͰ͖Δ Listen, Acceptͯ͠ConnΛgoroutineͰॲཧ͢Δ͚ͩ ln,

    err := net.Listen("tcp", ":8080") if err != nil { // handle error } for { conn, err := ln.Accept() if err != nil { // handle error } go handleConnection(conn) }
  5. Go Ͱ TCP Echo αʔόɺ௒؆୯ func handleConnection(conn net.Conn) { defer

    conn.Close() // ൈ͚Δͱ͖ʹ੾அ b := bufio.NewReader(conn) for { line, err := b.ReadString('\n') // 1ߦಡΉ if err != nil { return } _, err = io.WriteString(conn, line) // Φ΢Ϝฦ͠ if err != nil { return } } }
  6. for { conn, err := ln.Accept() if err != nil

    { // handle error } conn.SetDeadline(time.Now().Add(idleTimeout)) go handleConnection(conn) } func handleConnection(conn net.Conn) { defer conn.Close() buf := bufio.NewReader(conn) for { line, _ := buf.ReadString('\n') // 1ߦಡΉ conn.SetDeadline(time.Now().Add(idleTimeout)) // ^λΠϜΞ΢τԆ௕
  7. ϓϩηεऴྃ࣌ʹ੾அ αʔό΋ͪΌΜͱ੾அ͠ͳ͍ͱߦّ͕ѱ͍ΑͶ? func (app *App) handleConn(ctx context.Context, conn net.Conn) {

    defer conn.Close() // ͜ͷίωΫγϣϯͷॲཧΛࣗൃతʹ੾Δͱ͖ go func() { <-ctx.Done() // ϓϩηε͕γάφϧΛड͚Δͱ͜ͷctx͕Done conn.Close() // ͪΌΜͱ੾Δʂ }() // ...
  8. ͜͜Ͱ໰୊Ͱ͢ ͜ͷίʔυͰ͸ͳʹ͔͕ϦʔΫ͠·͢ func (app *App) handleConn(ctx context.Context, conn net.Conn) {

    defer conn.Close() // ͜ͷίωΫγϣϯͷॲཧΛࣗൃతʹ੾Δͱ͖ go func() { <-ctx.Done() // ϓϩηε͕γάφϧΛड͚Δͱ͜ͷctx͕Done conn.Close() // ͪΌΜͱ੾Δʂ }() // ...
  9. goroutineϦʔΫ handleConn͕returnͯ͠΋goroutine͕ࢮͳͳ͍ func (app *App) handleConn(ctx context.Context, conn net.Conn) {

    defer conn.Close() // ͜ͷίωΫγϣϯͷॲཧΛࣗൃతʹ੾Δͱ͖ go func() { <-ctx.Done() // ͜Ε͕ݺ͹ΕΔ(=ϓϩηεऴྃ)·Ͱ conn.Close() // goroutine ͕ੜ͖࢒Δ }() // ...
  10. मਖ਼ํ๏1 ࢠ context Λ࡞ͬͯ defer cancel() ࣗൃతऴྃ࣌ɺ਌ऴྃ࣌ͲͪΒͰ΋ίωΫγϣϯ੾அͱ goroutineऴ͕ྃͰ͖ΔͷͰϦʔΫ͠ͳ͍ func (app

    *App) handleConn(ctx context.Context, conn net.Conn) { ctx2, cancel := context.WithCancel(ctx) defer cancel() go func() { <-ctx2.Done() conn.Close() }() // ... 1 Fix goroutine leak. #28 https://github.com/kayac/go-katsubushi/pull/28/files
  11. ʮ͜ͷ katsubushi ಈ͍ͯΔ͔ͳ?ʯ $ curl localhost:11212 VALUE / 0 18

    591644124189298688 VALUE HTTP/1.1 0 18 591644124189298689 END ERROR ERROR ERROR ʮID͸ฦ͖͚ͬͯͨͲͳʹ͜Ε??ʯ ʮ͋ɺcurlͰୟ͍ͯͨɻɻɻʯ
  12. Կ͕ى͖͍͔ͯͨ curl ͸ katsubushi ʹ HTTP ͰϦΫΤετͨ͠(౰વ) $ curl -v

    127.0.0.1:11212 > GET / HTTP/1.1 > Host: 127.0.0.1:11212 > User-Agent: curl/7.65.1 > Accept: */* > (ۭߦ) HTTPͷϦΫΤετ͸ memcached protocol Ͱ΋ղऍͰ͖Δʂ memcached protocol Ͱ஋ͷऔಘ: GET key1 key2 HTTP Ͱ GET ϦΫΤετͷ1ߦ໨: GET / HTTP/1.1
  13. ͭ·Γ͜͏͍͏Ϩεϙϯε͕ฦΔ VALUE / 0 18 # / ͱ͍͏keyͷ஋(flag=0, 18byte) 591644124189298688

    VALUE HTTP/1.1 0 18 # HTTP/1.1 ͱ͍͏keyͷ஋(flag=0, 18byte) 591644124189298689 END # HTTPͷ1ߦ໨ʹ͸Ϩεϙϯε͕ฦͤͨΑʂ ERROR # Host: ͱ͔஌Βͳ͍! ERROR # User-Agnet: ͱ͔஌Βͳ͍!! ERROR # Accept: ͱ͔஌Βͳ͍!!!
  14. Ұํͦͷ͜Ζ HTTP Λ৯΂ͨ katsubushi ͸ panic: runtime error: invalid memory

    address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x134807f] goroutine 20 [running]: github.com/kayac/go-katsubushi.(*App).handleConn(0xc00012a000, 0x14a3de0, 0xc0000ac038) /Users/fujiwara/src/github.com/kayac/go-katsubushi/app.go:160 +0x2cf created by github.com/kayac/go-katsubushi.(*App).Listen /Users/fujiwara/src/github.com/kayac/go-katsubushi/app.go:131 +0x2ed ʮ͠ɺࢮΜͰΔ…ʯ
  15. ൜ਓ func (app *App) BytesToCmd(data []byte) (cmd MemdCmd, err error)

    { if len(data) == 0 { // ۭߦͳͷͰ 0 byte return nil, nil // ← ͍ͭ͜ } foo, err := Foo() ͱ͍͏ΠϯλʔϑΣʔεΛΈͨ ී௨ͷGopher͕ߟ͑Δ͜ͱ ʮerr != nil ͳΒ foo == nil ͔΋͔ͩΒؾΛ͚ͭΑ͏ʯ ʮerr == nil ͳΒ foo != nil ͩΑͶʂʯ
  16. ରԠ2 func (app *App) BytesToCmd(data []byte) (cmd MemdCmd, err error)

    { if len(data) == 0 { - return nil, nil + return nil, errors.New("No command") ஌Βͳ͍ίϚϯυ͕དྷͨΒଈ࠲ʹ઀ଓΛ੾ͬͨ΄͏͕͍͍͔΋ 2 Never die #8 https://github.com/kayac/go-katsubushi/pull/8
  17. katsubushi ID ͸࣌ࠁϕʔε +-+-----------------------------------------+----------+------------+ | | Timestamp | WorkerID |

    Sequence | +-+-----------------------------------------+----------+------------+ |0|00001001001001010100110010111011011100100|0000000001|000000000000| +-+-----------------------------------------+----------+------------+ | | 78,560,982,756ms since epoch | 1| 0| +-+-----------------------------------------+----------+------------+ = 2017-06-28T06:29:42.756Z ࣌ࠁ͕໭Δͱಉ͡ID͕ൃߦ͞ΕΔՄೳੑ͕͋Δ
  18. monotonic time https://golang.org/doc/go1.9#monotonic-time Go 1.9ͰOS࣌ࠁ͕໭ͬͯ΋ time.Now() ͸໭Βͳ͍ monotonic time͕࣮૷͞Εͨ 2017−01−01ͷӞඵൃੜ࣌

    Cloudflare͕࣌ࠁͷר͖໭ΓͰো֐3 ࣌ࠁ͕໭ͬͨ݁Ռ math/rand.Int63n ʹෛͷ஋͕౉Γpanic 3 How and why the leap second affected Cloudflare DNS https://blog.cloudflare.com/how-and-why-the-leap-second-affected- cloudflare-dns/
  19. monotonic time Ҏલͷ katsubushi ts := g.timestamp() // for rewind

    of server clock if ts < g.lastTimestamp { return 0, errors.New("system clock was rollbacked") } // ུ g.lastTimestamp = ts ࣌ࠁ͕ר͖໭ͬͨΒΤϥʔʹͯ͠๷ޚ ID͕ඃΔ͙Β͍ͳΒൃߦͰ͖ͳ͍ํ͕Ϛγ
  20. monotonic time ରԠ4 katsubushi ID ͷ࣌ࠁ෦෼͸ 2015-01-01T00:00:00 UTC Λىݯͱͨ͠ܦաඵ਺ var

    Epoch = time.Date(2015, 1, 1, 0, 0, 0, 0, time.UTC) d := time.Now().Sub(Epoch) ͜ͷΑ͏ʹɺ೔࣌ࢦఆͰ࡞ͬͨ஋͔Βܭࢉ͢Δͱmonotonicʹ ͳΒͳ͍ 4 use Monotonic time #21 https://github.com/kayac/go-katsubushi/pull/21
  21. monotonic time ରԠ now := time.Now() startedAt := now offset

    := now.Sub(Epoch) // ىಈ࣌ͷ࣌ࠁͱepochͷࠩΛܭࢉ͓ͯ͘͠ // ... d := time.Now().Sub(startedAt) + offset time.Now() Ͱ࡞ͬͨ஋ಉ࢜Λൺֱͯ͠offsetΛௐ੔͢Ε͹ ר͖໭Βͳ͍
  22. ࣌ࠁͷଞʹ΋ඃΔՄೳੑ͕͋Δ෦෼͕… +-+-----------------------------------------+----------+------------+ | | Timestamp | WorkerID | Sequence |

    +-+-----------------------------------------+----------+------------+ |0|00001001001001010100110010111011011100100|0000000001|000000000000| +-+-----------------------------------------+----------+------------+ | | 78,560,982,756ms since epoch | 1| 0| +-+-----------------------------------------+----------+------------+ = 2017-06-28T06:29:42.756Z WorkerID = ಉ࣌ʹಈ࡞͍ͯ͠ΔϓϩηεؒͰҰҙͷID