Real World Migration from HTTP to gRPC in Ruby #grpcconf

This is a slide I presented at gRPC Conf 2020 titled "Real World Migration from HTTP to gRPC in Ruby".

In this talk, I presented the details of our migration approach from HTTP/1.1 to gRPC, and the resulting performance improvements. Average latency got decreased by 50%, and 90%tile latency was decreased to a seventh.

cf. https://sched.co/cRfr

Transcript

1. ©2020 Wantedly, Inc. Real World Migration from HTTP to gRPC in Ruby 27.Jul.2020 - Nao Minami (@south37) Wantedly, Inc. gRPC Conf 2020
2. ©2020 Wantedly, Inc. /BP
   .JOBNJ *OGSBTUSVDUVSF
   5FBN
   8BOUFEMZ
   *OD  43&
   4JUF
   3FMJBCJMJUZ
   &OHJOFFSJOH
    %FWFMPQNFOU
   *OGSBTUSVDUVSF
    .JDSPTFSWJDFT
   H31$
   FUD
3. ©2020 Wantedly, Inc. 1. Why gRPC 2. How we migrated from HTTP to gRPC 3. What we achieved by using gRPC In this talk…
4. ©2020 Wantedly, Inc. 1. Why gRPC
5. ©2020 Wantedly, Inc. Background: our products Our products w 8BOUFEMZ
   7JTJU
   
   KPCT
   BOE
   DBOEJEBUFT
   NBUDIJOH
   w 8BOUFEMZ
   1FPQMF
   
   QSPGFTTJPOBM
   DPOUBDUT
   NBOBHFNFOU
   w FUD
   These products share the same components such as “user’s profile data”
6. ©2020 Wantedly, Inc. Background: our microservices architecture Our products are built with 100+ microservices w 
   NJDSPTFSWJDFT
   XSJUUFO
   JO
   3VCZ
   (P
   1ZUIPO
   /PEFKT
   FUD
   SVO
   PO
   B
   LT
   DMVTUFS
   w 8F
   IBE
   CFFO
   VTJOH
   )551
   GPS
   UIF
   NBKPSJUZ
   PG
   PVS
   NJDSPTFSWJDFT
   w lVTFS`T
   QSPpMF
   EBUBz
   JT
   NBOBHFE
   CZ
   l6TFS4FSWJDFz
   XIJDI
   JT
   XSJUUFO
   JO
   3VCZ LT
   DMVTUFS $PSF
   4FSWJDFT 6TFS4FSWJDF
   3VCZ
7. ©2020 Wantedly, Inc. What is “UserService”? Get/Update user’s profile data Low latency and high throughput are required w 6TFE
   GSPN
   NBOZ
   NJDSPTFSWJDFT
   
   IJHI
   UISPVHIQVU
   JT
   SFRVJSFE
   w -BUFODZ
   PGUFO
   EJSFDUMZ
   B⒎FDUT
   UIF
   VTFS
   FYQFSJFODF
   
   MPX
   MBUFODZ
   JT
   SFRVJSFE 6TFS4FSWJDF (FUVQEBUF
   VTFS`T
   QSPpMF
   EBUB
8. ©2020 Wantedly, Inc. What was the problem of UserService? Latency caused by using HTTP/1.1 w 8JUIPVU
   LFFQBMJWF
   )551
   EPFT
   OPU
   QFSGPSN
   XFMM
   GPS
   TFWFSBM
   SFBTPOT
   w FH
   %/4
   MPPLVQ
   5$1
   XBZ
   IBOETIBLF
   5$1
   TMPXTUBSU
   FUD 6TFS4FSWJDF -BUFODZ
   DBVTFE
   CZ
   VTJOH
   )551
   
9. ©2020 Wantedly, Inc.
10. ©2020 Wantedly, Inc. Why gRPC? High-Performance w -POHMJWFE
    SFBMUJNF
    DPNNVOJDBUJPO
    TUSFBNT
    CZ
    VTJOH
    )551
    w -PX
    MBUFODZ
    IJHI
    UISPVHIQVU
    Protocol Buffers w l4JOHMF
    TPVSDF
    PG
    USVUIz
    PG
    B
    TFSWJDF
    EFpOJUJPO
    w "EEJUJPOBM
    HPPE
    QPJOU
    XF
    XFSF
    BMSFBEZ
    VTJOH
    1SPUPDPM
    #V⒎FST
    PWFS
    )551
    Multiple Languages Support w 8F
    IBWF
    NJDSPTFSWJDFT
    XSJUUFO
    JO
    TFWFSBM
    MBOHVBHFT
    BOE
    H31$
    TVQQPSUT
    BMM
    PG
    UIFN
11. ©2020 Wantedly, Inc. For these reasons, we chose gRPC
12. ©2020 Wantedly, Inc. 2. How we migrated from HTTP to gRPC
13. ©2020 Wantedly, Inc. 1. Preparation 2. Migration How we migrated from HTTP to gRPC
14. ©2020 Wantedly, Inc. 1. Preparation 2. Migration How we migrated from HTTP to gRPC
15. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive Development Environment 4. Load Balancing Preparations for using gRPC
16. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive Development Environment 4. Load Balancing Preparations for using gRPC
17. ©2020 Wantedly, Inc. Without a monorepo, we have to share the same proto files across the multiple repositories Schema Management 4BNF
    QSPUP
    pMFT 3FQPTJUPSZ
    " 3FQPTJUPSZ
    # 3FQPTJUPSZ
    $
18. ©2020 Wantedly, Inc. How should we share proto files? w $PQZ
    
    QBTUF
    NBOVBMMZ
    
    
    
    
    7FSZ
    IBSE
    UP
    VQEBUF
    NVMUJQMF
    SFQPTJUPSJFT
    DPOUJOVPVTMZ
    w %PXOMPBE
    QSPUP
    pMFT
    BVUPNBUJDBMMZ
    
    
    
    
    
    /FFE
    UP
    HFOFSBUF
    UIF
    DPEF
    NBOVBMMZ
    
    
    
    
    4V⒎FS
    GSPN
    UIF
    EJ⒎FSFODFT
    JO
    FBDI
    EFWFMPQFS`T
    FOWJSPONFOU
    FH
    WFSTJPO
    PG
    
    
    
    
    
    
    
    
    
    QSPUPD
    QSFTFODF
    PG
    QMVHJOT
    FUD
    We decided to make a central repository Schema Management
19. ©2020 Wantedly, Inc. apis: a central repository for schema management w lBQJTz
    TUPSFT
    BMM
    QSPUP
    pMFT
    GPS
    PVS
    DPNQBOZ Schema Management
20. ©2020 Wantedly, Inc. How to use “apis” repository? Code is generated in the CI pipelines of “apis” w 3VCZ
    (P
    BOE
    /PEFKT
    DPEF
    JT
    HFOFSBUFE
    GPS
    FBDI
    QSPUP
    pMF
    w (FOFSBUFE
    DPEF
    JT
    EJTUSJCVUFE
    WJB
    QBDLBHF
    NBOBHFST
    #VOEMFS
    (P
    .PEVMFT
    OQN
    w %FWFMPQFST
    EPOU
    OFFE
    UP
    HFOFSBUF
    DPEF
    CZ
    UIFNTFMWFT
21. ©2020 Wantedly, Inc. # Gemfile gem 'apis', git: 'git@github.com:wantedly/apis-ruby.git', tag: 'xxx.yyy.zzz' 3VCZ
    XJUI
    #VOEMFS
22. ©2020 Wantedly, Inc. How we develop gRPC servers and clients in Ruby?
23. ©2020 Wantedly, Inc. gRPC servers: Implement a class which inherits a gRPC service class fetched from “apis” w #Z
    JNQMFNFOUJOH
    JOTUBODF
    NFUIPET
    GPS
    FBDI
    31$
    NFUIPE
    XF
    DBO
    VTF
    H31$
    TFSWFST
    JO
    3VCZ # NOTE: Load UsersPb::UserService from apis require "wantedly/users/users_services_pb" class UserServer < UsersPb::UserService::Service # @param [UsersPb::ListUsersRequest] req # @param [GRPC::ActiveCall::SingleReqView] call # @return [UsersPb::ListUsersResponse] def list_users(req, call) svc = UsersReadService.new(req) UsersPb::ListUsersResponse.new(users: svc.users) end end How we develop gRPC servers?
24. ©2020 Wantedly, Inc. gRPC clients: Use a stub class fetched from “apis” w #Z
    VTJOH
    B
    TUVC
    PCKFDU
    BT
    B
    H31$
    DMJFOU
    XF
    DBO
    DPNNVOJDBUF
    XJUI
    H31$
    TFSWFST # NOTE: Load UsersPb::UserService from apis require "wantedly/users/users_services_pb" client = UsersPb::UserService::Stub.new( url, :this_channel_is_insecure ) How we develop gRPC clients?
25. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive Development Environment 4. Load Balancing Preparations for using gRPC
26. ©2020 Wantedly, Inc. Application monitoring is required in production w 8F
    XFSF
    VTJOH
    %BUBEPH
    /FX
    3FMJD
    )POFZCBEHFS
    FUD
    GPS
    NPOJUPSJOH
    PVS
    
    
    
    
    
    )551
    TFSWFST
    
    
    
    
    8BOUFE
    UP
    DPOUJOVF
    UP
    VTF
    UIFN
    GPS
    H31$
    TFSWFST Monitoring
27. ©2020 Wantedly, Inc. Monitoring How to monitor gRPC servers? => gRPC interceptors w /FFE
    UP
    TFOE
    NFUSJDT
    PG
    FBDI
    H31$
    SFRVFTU
    w 8JUI
    H31$
    JOUFSDFQUPST
    XF
    DBO
    JNQMFNFOU
    TVDI
    GFBUVSFT $MJFOU
    H31$
    JOUFSDFQUPST $MJFOU 4FSWFS
    H31$
    JOUFSDFQUPST 4FSWFS 3FRVFTU 3FTQPOTF
28. ©2020 Wantedly, Inc. Monitoring We implemented some gRPC interceptors w H31$
    JOUFSDFQUPST
    GPS
    FBDI
    4BB4
    QSPEVDU
    BSF
    OPU
    P⒏DJBMMZ
    QSPWJEFE
    BU
    UIJT
    UJNF
    w 8F
    JNQMFNFOUFE
    UIFN
    JO
    3VCZ
    BOE
    NBEF
    UIFN
    BWBJMBCMF
    UP
    UIF
    3VCZ
    DPNNVOJUZ
    BT
    PQFO
    TPVSDF
    TPGUXBSF
    044
29. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive Development Environment 4. Load Balancing Required preparations for using gRPC
30. ©2020 Wantedly, Inc. Need to create a productive development environment for gRPC w 'PS
    )551
    TFSWFST
    UIF
    FDPTZTUFN
    JT
    NBUVSF
    BOE
    NBOZ
    PQFO
    TPVSDF
    MJCSBSJFT
    FYJTU
    w 5IJT
    JT
    OPU
    UIF
    DBTF
    XJUI
    H31$
    TFSWFST
    FTQFDJBMMZ
    JO
    3VCZ
    8F
    OFFE
    UP
    DSFBUF
    B
    QSPEVDUJWJUZ
    EFWFMPQNFOU
    FOWJSPONFOU
    CZ
    PVSTFMWFT Productive Development Environment
31. ©2020 Wantedly, Inc. How to create a productive development environment?
32. ©2020 Wantedly, Inc. Use a common utility library named “servicex” w 8F
    XBOU
    TPNF
    VTFGVM
    GFBUVSFT
    FH
    IPUSFMPBEJOH
    VUJMJUZ
    GPS
    QSPUPDPM
    CV⒎FST
    FUD
    w 8F
    XBOU
    UP
    BWPJE
    SFQFBUJOH
    UIF
    TBNF
    DPOpHVSBUJPO
    JO
    NVMUJQMF
    NJDSPTFSWJDFT
    
    
    
    
    
    JNQMFNFOUFE
    lTFSWJDFYz How to create a productive development environment? .JDSPTFSWJDF
    " .JDSPTFSWJDF
    # .JDSPTFSWJDF
    $ lTFSWJDFYz
33. ©2020 Wantedly, Inc. gRPC servers: grpc-server command w H31$
    TFSWJDF
    DMBTTFT
    BSF
    BVUPNBUJDBMMZ
    MPBEFE
    w 4PNF
    H31$
    JOUFSDFQUPST
    BSF
    BVUPNBUJDBMMZ
    TFU
    w )PUSFMPBEJOH
    GFBUVSF
    w %FWFMPQFST
    DBO
    KVTU
    GPDVT
    PO
    UIF
    JNQMFNFOUBUJPO
    PG
    H31$
    TFSWJDF
    DMBTTFT The features of our internal library “servicex” $ bundle exec grpc-server handling /grpc.health.v1.Health/Check with #<Method: Grpc::Health::Checker#check> handling /grpc.health.v1.Health/Watch with #<Method: Grpc::Health::Checker(Grpc::Health::V1::Health::Service)#watch> handling /wantedly.users.UserService/ListUsers with #<Method: UsersGrpcService#list_users> . . . gRPC server starting... * Listening on tcp://0.0.0.0:6046 * Environment: development
34. ©2020 Wantedly, Inc. gRPC clients: Servicex::Grpc.stub_for w 4PNF
    H31$
    JOUFSDFQUPST
    BSF
    BVUPNBUJDBMMZ
    TFU
    w 6TFS"HFOU
    JT
    BVUPNBUJDBMMZ
    DPOpHVSFE
    w %FWFMPQFST
    DBO
    KVTU
    GPDVT
    PO
    JNQMFNFOUJOH
    UIF
    CVTJOFTT
    MPHJD The features of our internal library “servicex” # NOTE: Load UsersPb::UserService from apis require "wantedly/users/users_services_pb" grpc_server_url = “xxx” client = Servicex::Grpc.stub_for( UsersPb::UserService, grpc_server_url, ) client.get_user(UsersPb::GetUserRequest.new(…))
35. ©2020 Wantedly, Inc. Pb: utility for using Protocol Buffers w 8JUI
    l1Cz
    XF
    DBO
    DSFBUF
    QSPUPCVG
    PCKFDUT
    NPSF
    FBTJMZ
    w FH
    1CUP@UJNFTUBNQ
    DBO
    DSFBUF
    B
    UJNFTUBNQ
    PCKFDU
    GSPN
    B
    4USJOH
    PCKFDU The features of our internal library “servicex” [1] pry(main)> Pb.to_timestamp("2019-05-15T00:00:00+09:00") => <Google::Protobuf::Timestamp: seconds: 1557846000, nanos: 0> 8JUI
    l1Cz [1] pry(main)> Google::Protobuf::Timestamp.new( [1] pry(main)* seconds: Time.parse("2019-05-15T00:00:00+09:00").to_i [1] pry(main)* ) => <Google::Protobuf::Timestamp: seconds: 1557846000, nanos: 0> 8JUIPVU
    l1Cz
36. ©2020 Wantedly, Inc. With Pb.to_proto, we can create user-defined protobuf objects easily The features of “Pb” [1] pry(main)> Pb.to_proto(Account, { created_at: "2019-05-15T00:00:00+09:00" }) => <Account: id: 0, created_at: <Google::Protobuf::Timestamp: seconds: 1557846000, nanos: 0>> 8JUI
    l1Cz [1] pry(main)> Account.new( [1] pry(main)* created_at: Google::Protobuf::Timestamp.new( [1] pry(main)* seconds: Time.parse("2019-05-15T00:00:00+09:00").to_i [1] pry(main)* ) [1] pry(main)* ) => <Account: id: 0, created_at: <Google::Protobuf::Timestamp: seconds: 1557846000, nanos: 0>> 8JUIPVU
    l1Cz
37. ©2020 Wantedly, Inc. Pb is published as open source software (OSS) w 1C
    JT
    JODMVEFE
    JO
    TFSWJDFY
    CVU
    XF
    NBEF
    JU
    BWBJMBCMF
    UP
    UIF
    3VCZ
    DPNNVOJUZ
    BT
    PQFO
    TPVSDF
    TPGUXBSF
    044 
    w IUUQTSVCZHFNTPSHHFNTUIF@QC
    w IUUQTSVCZHFNTPSHHFNTQCTFSJBMJ[FS The features of “Pb”
38. ©2020 Wantedly, Inc. Use gRPC combined with parts of Ruby on Rails w 8F
    NJHSBUFE
    GSPN
    3VCZ
    PO
    3BJMT
    BOE
    XBOUFE
    UP
    LFFQ
    VTJOH
    QBSUT
    PG
    GFBUVSFT
    JO
    JU
    FH
    "DUJWF3FDPSE
    "DUJWF4VQQPSU
    "VUPMPBEJOH
    SBJMT
    DPOTPMF
    FUD
    
    
    
    
    
    %FDJEFE
    UP
    VTF
    HSQD
    HFN
    XJUI
    SBJMT
    HFN
    w 8F
    GPVOE
    UIBU
    JU`T
    WFSZ
    VTFGVM
    UP
    CVJME
    B
    H31$
    TFSWFS
    XJUI
    QBSUT
    PG
    3VCZ
    PO
    3BJMT
    8F
    SFDPNNFOE
    UIJT
    DPOpHVSBUJPO
    JG
    ZPV`SF
    JO
    UIF
    TBNF
    TJUVBUJPO # Gemfile ruby '2.7.1' gem 'grpc' gem 'rails', '~> 6.0.3' How to create a productive development environment?
39. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive Development Environment 4. Load Balancing Required preparations for using gRPC
40. ©2020 Wantedly, Inc. For gRPC, L4 Load-Balancers aren’t very useful w H31$
    JT
    CVJMU
    PO
    )551
    XIJDI
    JT
    EFTJHOFE
    UP
    IBWF
    B
    TJOHMF
    MPOHMJWFE
    5$1
    DPOOFDUJPO
    w *O
    -
    -PBE#BMBODFST
    PODF
    UIF
    5$1
    DPOOFDUJPO
    JT
    FTUBCMJTIFE
    BMM
    SFRVFTUT
    XJMM
    HFU
    QJOOFE
    UP
    B
    TJOHMF
    EFTUJOBUJPO Load Balancing 6TFS4FSWJDF 4PNF
    .JDSPTFSWJDF
41. ©2020 Wantedly, Inc. We use Istio for L7 load-balancing in k8s clusters w &OWPZ
    DPOUBJOFST
    DPOpHVSFE
    CZ
    *TUJP
    BSF
    JOKFDUFE
    BT
    TJEFDBS
    QSPYJFT
    BOE
    VTFE
    GPS
    -
    "QQMJDBUJPO
    MPBECBMBODJOH Load Balancing 6TFS4FSWJDF 4PNF
    .JDSPTFSWJDF &OWPZ
42. ©2020 Wantedly, Inc. 1. Preparation 2. Migration How we migrated from HTTP to gRPC
43. ©2020 Wantedly, Inc. Basic strategy was “Try Small” w "U
    pSTU
    XF
    NBEF
    B
    QSPUPUZQF
    BT
    B
    QSPPG
    PG
    DPODFQU
    1P$
    w /FYU
    XF
    VTFE
    H31$
    GPS
    B
    QBSU
    PG
    UIF
    USB⒏D
    JO
    QSPEVDUJPO
    BOE
    HSBEVBMMZ
    JODSFBTFE
    UIF
    QFSDFOUBHF How we migrated from HTTP to gRPC?
44. ©2020 Wantedly, Inc. PoC - schematic diagram QSPEVDUJPO
    LT
    DMVTUFS
    6TFS4FSWJDF )551 Tried PoC in development k8s cluster w #Z
    VTJOH
    lEFWFMPQNFOU
    LT
    DMVTUFSz
    XF
    DPVME
    USZ
    H31$
    XJUIPVU
    XPSSZJOH
    BCPVU
    B⒎FDUJOH
    QSPEVDUJPO
    FOWJSPONFOU EFWFMPQNFOU
    LT
    DMVTUFS
    6TFS4FSWJDF 5SZ
    H31$
45. ©2020 Wantedly, Inc. The result of PoC Latency decreased dramatically w #Z
    TFFJOH
    UIJT
    SFTVMU
    XF
    EFDJEFE
    UP
    USZ
    H31$
    JO
    B
    QSPEVDUJPO
    LT
    DMVTUFS %ile Latency [ms]
46. ©2020 Wantedly, Inc. How we tried gRPC in production?
47. ©2020 Wantedly, Inc. How we tried gRPC in production? At first, we used gRPC for a small part of the traffic w #Z
    USZJOH
    TNBMM
    XF
    DPVME
    NJOJNJ[F
    UIF
    QSFQBSBUJPO
    DPTUT
    UP
    NJOJNJ[F
    UIF
    SJTL
    Then, gradually increased the percentage w 'JOBMMZ
    XF
    DPVME
    BDIJFWF
    UIF
    DPNQMFUF
    NJHSBUJPO )551
     6TFS
    H31$
     *ODSFBTF
    UIF
    QFSDFOUBHF )551
     6TFS
    H31$
    
48. ©2020 Wantedly, Inc. With these steps, we migrated to gRPC
49. ©2020 Wantedly, Inc. 3. What we achieved by using gRPC
50. ©2020 Wantedly, Inc. Results achieved by using gRPC Average Latency decreased w "WFSBHF
    MBUFODZ
    EFDSFBTFE
    CZ
    BCPVU
    NT
    QFS
    SFRVFTU
    w 5IF
    JNQBDU
    XBT
    NPSF
    SFNBSLBCMF
    XIFO
    NBOZ
    SFRVFTUT
    UP
    6TFS4FSWJDF
    XFSF
    JOWPMWFE
    'PS
    FYBNQMF
    JO
    TPNF
    NJDSPTFSWJDFT
    BWFSBHF
    MBUFODZ
    EFDSFBTFE
    CZ
    B
    IBMG HTTP/1.1 gRPC Average latency decreased from 130ms to 65ms in this service
51. ©2020 Wantedly, Inc. We achieved huge performance improvement
52. ©2020 Wantedly, Inc. What we have been doing since the migration?
53. ©2020 Wantedly, Inc. The use of gRPC is increasing throughout the organization w "MNPTU
    BMM
    DMJFOUT
    PG
    6TFS4FSWJDF
    BSF
    VTJOH
    H31$
    w /FX
    NJDSPTFSWJDFT
    BSF
    CFJOH
    EFWFMPQFE
    VTJOH
    H31$
    w H31$
    TUBSUFE
    UP
    CF
    VTFE
    JO
    PUIFS
    MBOHVBHFT
    (P
    /PEFKT
    FUD What we have been doing since the migration? 3VCZ 3VCZ /PEF 3VCZ (P H31$ /FX
54. ©2020 Wantedly, Inc. Summary
55. ©2020 Wantedly, Inc. Summary 1. Why gRPC w )JHI1FSGPSNBODF
    w 1SPUPDPM
    #V⒎FST
    w .VMUJQMF
    -BOHVBHFT
    4VQQPSU
    2. How we migrated from HTTP to gRPC w $SFBUFE
    B
    XPSLqPX
    GPS
    1SPUPDPM
    #V⒎FST
    VTJOH
    PVS
    lBQJTz
    SFQPTJUPSZ
    w %FWFMPQFE
    MJCSBSJFT
    GPS
    QSPEVDUJWJUZ
    BOE
    QVCMJTIFE
    TPNF
    PG
    UIFN
    BT
    044
    w #BTJD
    TUSBUFHZ
    GPS
    NJHSBUJPO
    XBT
    l5SZ
    4NBMMz
    3. What we achieved by using gRPC w "DIJFWFE
    UIF
    QFSGPSNBODF
    JNQSPWFNFOU
    
    w 5IF
    VTF
    PG
    H31$
    JT
    JODSFBTJOH
    UISPVHIPVU
    UIF
    PSHBOJ[BUJPO
56. ©2020 Wantedly, Inc. • Alina Grubnyak https://unsplash.com/photos/hdpDLIXZg7Y Photo Credit