Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DroidKaigi 2018 gRPC/Protobuf

k-mats
February 07, 2018
5.8k

DroidKaigi 2018 gRPC/Protobuf

k-mats

February 07, 2018
Tweet

Transcript

  1. Outline • RPC ͱ͸ • gRPC ͱ͸ • Protocol Buffers

    ͱ͸ • gRPC + Protocol Buffers ͷಋೖ • ։ൃ্ͷ޻෉ • protoϑΝΠϧͷڞ༗ํ๏ͷྫ
  2. RPC > Remote procedure call is the synchronous language-level transfer

    of control between programs in disjoint address spaces whose primary communication medium is a narrow channel. ଞͷΞυϨεۭؒʢe.g. ωοτϫʔΫ্ͷαʔόʣʹ͋ΔॲཧΛ ࣮ߦ͢Δ͜ͱɻ Bruce Jay Nelson. 1981. Remote Procedure Call. Ph.D. Dissertation. Carnegie Mellon Univ., Pittsburgh, PA, USA. AAI8204168. https://dl.acm.org/citation.cfm?id=910306
  3. REST vs RPC as HTTP API • ҧ͍ͷ1ͭ:ʮAPIઃܭ࣌ʹͲ͜ʹண໨͢Δ͔ʯ • Ϧιʔεʢૢ࡞ର৅ʣͱΞΫγϣϯʢૢ࡞ํ

    ๏ʣΛͲ͏΍ͬͯදݱ͢Δ͔ • REST: ϦιʔεΛத৺ʹߟ͑Δ • RPC: ΞΫγϣϯΛத৺ʹߟ͑Δ
  4. RESTͷํ͕޲͍͍ͯΔέʔε • e.g. ϒϩάαʔϏεͷAPI • APIͷओͳ໾ׂ͸ϒϩάهࣄʹରͯ͠CRUD ૢ࡞Λ͢Δ͜ͱ • هࣄͱ͍͏ϦιʔεΛத৺ʹߟ͑ͨํ͕ྑ͍ •

    ΞΫγϣϯ͸HTTPϝιουͰදݱ https://www.smashingmagazine.com/2016/09/understanding-rest-and-rpc-for-http-apis/
  5. ϒϩάAPI: RESTͷ৔߹ • Create: • POST https://example.com/api/articles • body: {“title”:

    “New post”, “author”: “John”, “description”: “…”} • Read: • GET https://example.com/api/articles/123 • Update: • PATCH https://example.com/api/articles/123 • body: {“title”: “Revised”} • Delete: • DELETE https://example.com/api/articles/123 https://www.smashingmagazine.com/2016/09/understanding-rest-and-rpc-for-http-apis/
  6. ϒϩάAPI: RPCͷ৔߹ • Create: • https://example.com/api/createArticle • body: {“title”: “New

    post”, “author”: “John”, “description”: “…”} • Read: • https://example.com/api/getArticle • Update: • https://example.com/api/updateArticle • body: {“article_id”: 123, “title”: “Revised”} • Delete: • https://example.com/api/deleteArticle https://www.smashingmagazine.com/2016/09/understanding-rest-and-rpc-for-http-apis/
  7. RPCͷํ͕޲͍͍ͯΔέʔε • e.g. SlackͷΑ͏ͳνϟοταʔϏεͷAPI • APIΛ௨ͯ͡ҎԼͷػೳΛఏڙ͍ͨ͠ͱ͢Δ • ͋Δνϟϯωϧʹ͍ΔϢʔβ͕νϟϯωϧ͔Βୀग़ ͢Δ (leave)

    • ͋Δνϟϯωϧʹ͍ΔϢʔβΛνϟϯωϧ͔Β௥͍ ग़͠ɺ࠶ࢀՃΛې͡Δʢkickʣ https://www.smashingmagazine.com/2016/09/understanding-rest-and-rpc-for-http-apis/
  8. νϟοτAPI: RESTͷ৔߹ • leaveͱkickͷҧ͍Λ៉ྷͳRESTͰදݱ͢Δͷ͕೉͍͠ • e.g. bodyʹԿ͔͠Βͷ৘ใΛ௥Ճ͢Δ౳ͷ޻෉͕ඞཁ • leave: •

    PATCH https://example.com/api/channels/123/users/456 • body: {“status”: “left”} • kick: • PATCH https://example.com/api/channels/123/users/456 • body: {“status”: “kicked”} https://www.smashingmagazine.com/2016/09/understanding-rest-and-rpc-for-http-apis/
  9. νϟοτAPI: RESTͷ৔߹ • ҎԼͷΑ͏ʹ͢ΔͱRESTͱRPCͷதؒͷΑ͏ͳ΋ͷʹͳͬͯ͠·͏ • leave: • POST https://example.com/api/channels/123/users/456/leave •

    kick: • POST https://example.com/api/channels/123/users/456/kick https://www.smashingmagazine.com/2016/09/understanding-rest-and-rpc-for-http-apis/
  10. νϟοτAPI: RPCͷ৔߹ • leave: • https://example.com/api/leaveFromChannel • body: {“channel_id”: 123,

    “user_id”: 456} • kick: • https://example.com/api/kickFromChannel • body: {“channel_id”: 123, “user_id”: 456} https://www.smashingmagazine.com/2016/09/understanding-rest-and-rpc-for-http-apis/
  11. Slackͷweb API͸RPC > The Web API is a collection of

    HTTP RPC-style methods, all with URLs in the form `https://slack.com/api/METHOD_FAMILY.method`. https://api.slack.com/web Web API͸HTTP RPCελΠϧͷϝιουͷू·ΓͰ͋ΓɺશͯҎ ԼͷϑΥʔϜͰද͞Ε·͢ɻ `https://slack.com/api/METHOD_FAMILY.method`.
  12. ͦͷଞͷAPI • Dropbox API v2ͷେ൒͸RPC • AWS APIͷҰ෦͸RPC • Discord

    API͸REST https://www.dropbox.com/developers/documentation/http/documentation https://docs.aws.amazon.com/Route53/latest/APIReference/requests-rpc.html https://github.com/discordapp/discord-api-docs/blob/master/docs/Reference.md
  13. ͍ͪAndroid։ൃऀ͔Βݟͨ gRPCͷσϝϦοτ • ੈͷதͷ৘ใ͕·ͩे෼Ͱ͸ͳ͍ • e.g. TLS 1.2͕ຊ౰ʹͪΌΜͱ࢖ΘΕ͍ͯΔͷ͔Λௐ΂ΔͨΊʹ grpc-javaͷιʔείʔυΛಡΜͩΓ •

    https://github.com/grpc/grpc-java/blob/master/ SECURITY.md ͳͲެࣜͷࢿྉ͸͋Γ·͢ • e.g. AndroidΫϥΠΞϯτ͸Java, GoͰॻ͔Εͨαʔόͱ͸௨৴Ͱ ͖ΔͷʹNode.js, RubyͰॻ͔Εͨαʔόͱ͸௨৴Ͱ͖ͳ͍ࣄ৅ʹ ૺ۰ͨ͠Γʢ2017೥ͷ࿩ʣ
  14. gRPCͷର߅അ”Twirp” • 2018೥1݄ެ։ͷTwitch੡RPCϑϨʔϜϫʔΫ • γϯϓϧ͔ͭHTTP/1.1Λ࢖͑ΔʢHTTP/2΋Մʣ • ࣮૷ʹProtobufΛ࢖͏͕ɺbodyͷΤϯίʔυܗࣜ͸ProtobufʢόΠφϦʣ͚ͩͰͳ͘ JSON΋Մ • ࣮͸Protobufࣗମ͕JSON΁ΤϯίʔσΟϯάͰ͖ΔʢJSON

    Mappingʣ • gRPCͰ͸ඇαϙʔτͷ໛༷ • ૒ํ޲ετϦʔϛϯά͸ඇαϙʔτ • Twitch੡͸GoɺαʔυύʔςΟ੡͸Java, Lua, JS, RubyͳͲ https://blog.twitch.tv/twirp-a-sweet-new-rpc-framework-for-go-5f2febbf35f https://groups.google.com/forum/#!topic/grpc-io/ncVgOlp16F8
  15. Protocol Buffers (Protobuf) • XML౳ͷΑ͏ʹߏ଄Խ͞Εͨσʔλදݱܗࣜɺ ͳ͍͠͸IDL (Interface Description Language) •

    ݱࡏͷ࠷৽͸ver.3 (proto3) > think XML, but smaller, faster, and simpler. https://developers.google.com/protocol-buffers/docs/overview
  16. .protoϑΝΠϧ syntax = "proto3"; message Bank { int32 code =

    1; string name = 2; string name_kana = 3; } https://developers.google.com/protocol-buffers/docs/overview
  17. Protobufͱ͸ • .protoϑΝΠϧΛαʔόɾΫϥΠΞϯτؒͰڞ༗ • Protobuf compilerʹΑͬͯ.protoϑΝΠϧ͔ΒγϦΞϥΠζɾσγϦΞϥΠ ζ༻ͷΫϥε͕ੜ੒͞ΕΔ • όΠφϦʹΤϯίʔυͯ͠ૹ৴ •

    JSON΁Τϯίʔυ͢Δ͜ͱ΋Մ • ͨͩ͠ઌड़ͷ௨ΓgRPCͰ͸ະαϙʔτɺTwirpͰ͸αϙʔτ͍ͯ͠Δ໛༷ • ެࣜɾαʔυύʔςΟ߹Θͤͯ10छྨҎ্ͷݴޠʹରԠ https://developers.google.com/protocol-buffers/docs/overview
  18. γϦΞϥΠζ (Java) Bank bank = Bank.newBuilder() .setCode(123) .setName("υϩΠυۜߦ") .setNameKana("υϩΠυΪϯί΢") .build();

    output = new FileOutputStream(path); bank.writeTo(output); https://developers.google.com/protocol-buffers/docs/overview
  19. σγϦΞϥΠζ (C++) Bank bank; fstream input(path, ios::in | ios::binary); bank.ParseFromIstream(&input);

    int code = bank.code(); string name = bank.name(); string nameKana = bank.nameKana(); https://developers.google.com/protocol-buffers/docs/overview
  20. ഑ྻͱmessageͷೖΕࢠ message GetBanksResponse { repeated Bank bank = 1; }

    message Bank { int32 code = 1; string name = 2; string name_kana = 3; } https://developers.google.com/protocol-buffers/docs/overview
  21. Enum enum BankAccountType { SAVING = 0; // ී௨ CHECKING

    = 1; // ౰࠲ } https://developers.google.com/protocol-buffers/docs/overview
  22. Enum enum BankAccountType { option allow_alias = true; SAVING =

    0; // ී௨ CHECKING = 1; // ౰࠲ CURRENT = 1; // ౰࠲ʢผ໊ʣ } https://developers.google.com/protocol-buffers/docs/overview
  23. @Deprecated message Bank { int32 code = 1; string name

    = 2; string name_kana = 3 [deprecated=true]; string name_katakana = 4; } https://developers.google.com/protocol-buffers/docs/overview
  24. RPC service BankListService { rpc GetBanks (GetBanksRequest) returns (GetBanksResponse); }

    message GetBanksRequest {} message GetBanksResponse { repeated Bank bank = 1; } https://developers.google.com/protocol-buffers/docs/overview
  25. ͍ͪAndroid։ൃऀ͔Βݟͨ ProtobufͷσϝϦοτ • όΠφϦͰ௨৴͞ΕΔͷͰhuman readableͰ͸ͳ͍ • ※ JSON Mappingػೳ͸͋Δ͕ɺgRPCͰ͸ඇαϙʔτͷ໛༷ •

    curl΍charlesͳͲͰσόοάͰ͖ͳ͍ɺϒϥ΢βͰΞΫηεͯ͠Ϩεϙϯ εΛݟΔ͜ͱ͕Ͱ͖ͳ͍ • ͪΌΜͱ௨৴Ͱ͖͍ͯΔ͔Ͳ͏͔Λ֬ೝ͢ΔͨΊʹ؆୯ͳΫϥΠΞϯτΛ ॻ͍͓ͯ͘ඞཁ͕͋Δ • αʔόɾΫϥΠΞϯτؒͷprotoϑΝΠϧΛྑ͍ײ͡ʹಉظ͢Δ࢓૊Έ͕ඞཁ • ҰྫΛޙड़͠·͢
  26. ஫: ඞͣެࣜͷREADMEΛ֬ೝ • gRPC-Java: • https://github.com/grpc/grpc-java • Android޲͚ͱͦͷଞͷJava؀ڥ޲͚Ͱ಺༰͕ҟͳΔͷͰࠞಉ͠ͳ͍ཁ ஫ҙ •

    Protobuf: • https://github.com/google/protobuf-gradle-plugin • αϯϓϧ: • https://github.com/grpc/grpc-java/tree/master/examples/android
  27. // build.gradle buildscript { ... dependencies { ... classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.3'

    } } // app/build.gradle apply plugin: ‘com.android.application’ apply plugin: 'com.google.protobuf' dependencies { ... compile 'io.grpc:grpc-okhttp:1.9.0' compile 'io.grpc:grpc-protobuf-lite:1.9.0' compile 'io.grpc:grpc-stub:1.9.0' }
  28. // app/build.gradle protobuf { protoc { artifact = ‘com.google.protobuf:protoc:3.5.1-1’ }

    plugins { javalite { artifact = "com.google.protobuf:protoc-gen-javalite:3.0.0" } grpc { artifact = “io.grpc:protoc-gen-grpc-java:1.9.0” } } generateProtoTasks { all().each { task -> task.plugins { javalite {} grpc { option 'lite' } } } } }
  29. service BankListService { rpc GetBanks (GetBanksRequest) returns (GetBanksResponse); } message

    GetBanksRequest {} message GetBanksResponse { repeated Bank bank = 1; }
  30. ManagedChannel channel = OkHttpChannelBuilder .forAddress("example.com", "443") .build(); BankListServiceStub stub =

    BankListServiceGrpc.newStub(channel); GetBanksRquest request = GetBanksRequest .newBuilder() .build(); stub.getBanks(request, new StreamObserver<GetBanksResponse>() { @Override public void onNext(GetBanksResponse response) { List<Bank> banks = response.getBankList() ... } @Override public void onError(Throwable throwable) { ... } @Override public void onCompleted() { ... } );
  31. ֤repoͷprotoϑΝΠϧͷಉظ proto files repo iOS app repo Android app repo

    API server repo proto files proto files proto files ྑ͍ײ͡ʹprofo files repoΛίϐʔ or ࢀর͍ͨ͠
  32. proto files repo Digdag GCP Cloud Functions iOS app repo

    Android app repo API server repo 1. PRϚʔδͰwebhookૹ৴ 2. DigdagϫʔΫϑϩʔIDΛ ɹࢦఆͯ͠ϫʔΫϑϩʔىಈ 3. ֤repoͷprofo filesΛ ɹߋ৽͢ΔPRΛ࡞੒ 4. PRͷࠩ෼ΛݟͯϚʔδʢखಈʣ
  33. proto files repo Digdag GCP Cloud Functions iOS app repo

    Android app repo API server repo 1. PRϚʔδͰwebhookૹ৴ 2. DigdagϫʔΫϑϩʔIDΛ ɹࢦఆͯ͠ϫʔΫϑϩʔىಈ 3. ֤repoͷprofo filesΛ ɹߋ৽͢ΔPRΛ࡞੒ 4. PRͷࠩ෼ΛݟͯϚʔδʢखಈʣ
  34. proto files repo Digdag GCP Cloud Functions iOS app repo

    Android app repo API server repo 1. PRϚʔδͰwebhookૹ৴ 2. DigdagϫʔΫϑϩʔIDΛ ɹࢦఆͯ͠ϫʔΫϑϩʔىಈ 3. ֤repoͷprofo filesΛ ɹߋ৽͢ΔPRΛ࡞੒ 4. PRͷࠩ෼ΛݟͯϚʔδʢखಈʣ
  35. exports.syncTheoProto = function syncTheoProto (req, res) { ... postData =

    { "workflowId": settings.workflowId, ... } req.write(JSON.stringify(postData)); req.end(); ... }; POSTϦΫΤετʹ༧Ί࡞͓͍ͬͯͨ DigdagϫʔΫϑϩʔͷIDΛ෇Ճ͢Δ
  36. proto files repo Digdag GCP Cloud Functions iOS app repo

    Android app repo API server repo 1. PRϚʔδͰwebhookૹ৴ 2. DigdagϫʔΫϑϩʔIDΛ ɹࢦఆͯ͠ϫʔΫϑϩʔىಈ 3. ֤repoͷprofo filesΛ ɹߋ৽͢ΔPRΛ࡞੒ 4. PRͷࠩ෼ΛݟͯϚʔδʢखಈʣ
  37. proto files repo Digdag GCP Cloud Functions iOS app repo

    Android app repo API server repo 1. PRϚʔδͰwebhookૹ৴ 2. DigdagϫʔΫϑϩʔIDΛ ɹࢦఆͯ͠ϫʔΫϑϩʔىಈ 3. ֤repoͷprofo filesΛ ɹߋ৽͢ΔPRΛ࡞੒ 4. PRͷࠩ෼ΛݟͯϚʔδʢखಈʣ
  38. ࠷ޙʹ… Swagger or GraphQL or gRPC??? • গͳ͘ͱ΋ݱ࣌఺ͰͲΕ͔͕ઈରతʹྑ͍ͱ͍͏͜ͱ͸ແͦ͞͏ • ൺֱهࣄ͕୔ࢁ͋Δ

    • e.g. https://brandur.org/api-paradigms • ͜ͷهࣄʹର͢Δٞ࿦: https://news.ycombinator.com/item?id=14003134 • DroidKaigi 2018Ͱ΋ൃද͞Εͨʂ • Swagger: https://speakerdeck.com/magiepooh/madaapiding-yi-guan-li- dexiao-hao-siterufalse-swaggerwoyong-itada-gui-mo-apurishi-dai-falseapiding- yi-guan-li-tokodozienereto • GraphQL: https://speakerdeck.com/gfx/subarasikigraphqlfalsesekaiheyoukoso
  39. ൃදͷ·ͱΊ • RESTͱRPC͸దࡐదॴ • gRPC͸GoogleൃͷRPCϑϨʔϜϫʔΫ • Protobuf͸ߏ଄Խ͞Εͨσʔλදݱܗࣜͳ͍͠͸IDL • gRPC +

    ProtobufͰίʔυΛࣗಈੜ੒ • ಋೖ࣌͸ඞͣ࠷৽ͷެࣜREADMEΛ֬ೝ • ProtobufΛαʔόɾΫϥΠΞϯτؒͰಉظ͢Δ࢓૊ΈΛ࡞Ζ͏ • Swagger΍GraphQLʹ͍ͭͯ΋ௐ΂ͯΈΑ͏