Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Managing gRPC with Wire

Managing gRPC with Wire

At Cash App, mobile and server engineers collaborate on Protobuf and gRPC schemas to define APIs between their services. To improve our experience doing so, we built Wire. From a small in-house protobuf generation library, Wire has today matured into our favourite tool to manage and generate gRPC messages and services. Wire is backed by a versatile Gradle plugin and is used on our Android app, our iOS app, and our microservices at Cash App.
The talk will introduce Wire’s basic features before diving into:

- What Proto 3 support brings to the table in terms of new types and JSON serialisation, and the difference between Wire and protoc,
- How Wire handles options and provides comfortable APIs to consume them,
- Details about Wire Gradle plugin internals, what steps are processed and how to configure them,
- How Wire helps managing protobuf dependencies across microservices.

This presentation will provide simple entry points for people starting with Wire, and actionable improvements for experienced users alike.

Benoît Quenaudon

April 26, 2022
Tweet

More Decks by Benoît Quenaudon

Other Decks in Programming

Transcript

  1. // service.proto message Request { optional Customer customer = 1;

    } message Customer { required string name = 1; optional int32 age = 2; } service CustomerService { rpc getCustomer(Request) returns (Response); }
  2. Identity if absent Strings → "" Bytes → empty bytes

    Bools → false Numerics → 0, 0L, 0f, 0.0 Enums → first defined value (which must be 0) Messages → null
  3. syntax = "proto2"; message MyMessage { required string a =

    1; optional string b = 2; } class MyMessage( val a: String, val b: String? = null ) : Message<Message, Builder>() {} →
  4. syntax = "proto3"; message MyMessage { string a = 1;

    optional string b = 2; } class MyMessage( val a: String = "", val b: String? = null ) : Message<Message, Builder>() {} →
  5. Any → com.squareup.wire.AnyMessage Duration → java.time.Duration Timestamp → java.time.Instant Struct

    → map<String, *> Wrappers → Boxed types for primitives (String?) Empty → kotlin.Unit New Types google.protobuf. ~ Generic placeholder ~ same as java.time.Duration ~ same as java.time.Instant ~ JSON Object ~ JSON representation of primitive types + nullability ~ For parameter-less RPCs
  6. message Request { // Only set when it is a

    Bitcoin request. optional BitcoinData bitcoin_data = 1; // Only set when it is a Stock request. optional StockData stock_data = 2; }
  7. message Request { // Only set when it is a

    Bitcoin request. optional BitcoinData bitcoin_data = 1; // Only set when it is a Stock request. optional StockData stock_data = 2; } message Request { // Will be either BitcoinData or StockData. optional google.protobuf.Any request_data = 1; } ↓
  8. Any → com.squareup.wire.AnyMessage Duration → java.time.Duration Timestamp → java.time.Instant Struct

    → map<String, *> Wrappers → Boxed types for primitives (String?) Empty → kotlin.Unit New Types google.protobuf.
  9. Any → com.squareup.wire.AnyMessage Duration → java.time.Duration Timestamp → java.time.Instant Struct

    → map<String, *> Wrappers → Boxed primitives (e.g. String?) Empty → kotlin.Unit New Types google.protobuf.
  10. val millis = System.currentTimeMillis() val protocResponse: Message.EndpointResponse if (isResponseStale(...)) {

    fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
  11. val millis = System.currentTimeMillis() val protocResponse = Message.EndpointResponse.newBuilder() .setValidUntil( Timestamp...

    ) .build() if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
  12. val millis = System.currentTimeMillis() val protocResponse = Message.EndpointResponse.newBuilder() .setValidUntil( Timestamp.newBuilder()

    .setSeconds(millis / 1000) .setNanos(((millis % 1000) * 1000000).toInt()) .build() ) .build() if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
  13. val millis = System.currentTimeMillis() val protocResponse = Message.EndpointResponse.newBuilder().build() if (isResponseStale(

    Instant.ofEpochSecond( protocResponse.validUntil.seconds, protocResponse.validUntil.nanos.toLong() ) )) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
  14. Wire val millis = System.currentTimeMillis() val wireResponse: EndpointResponse if (isResponseStale(...))

    { fetch() } private fun isResponseStale(validUntil: Instant): Boolean
  15. Wire val millis = System.currentTimeMillis() val wireResponse = EndpointResponse(Instant.ofEpochMilli(millis)) if

    (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean
  16. Wire val millis = System.currentTimeMillis() val wireResponse = EndpointResponse(Instant.ofEpochMilli(millis)) if

    (isResponseStale(wireResponse.valid_until!!)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean
  17. Proto2 㲗 Proto3 Compatibility • Cannot reference proto2 enums in

    proto3 message. • Because of identity-if-absent. • Anything else is good.
  18. Protobuf Options message Money { optional int64 amount = 1

    [(squareup.redacted) = true]; optional string title = 2 [deprecated = true]; optional Currency current = 3 [(whatever.opt) = 3]; }
  19. Protobuf Options message Money { optional int64 amount = 1

    [(squareup.redacted) = true]; optional string title = 2 [deprecated = true]; optional Currency current = 3 [(whatever.opt) = 3]; }
  20. Protobuf Options • No intrinsic logic embedded • Wire handles

    a few • deprecated • default • json_name • packed • *.redacted • java_package • wire_package • "How about my option?"
  21. extend google.protobuf.MessageOptions { optional string documentation_url = 22200; } message

    Octagon { option (documentation_url) = "https://en.wikipedia.org/wiki/Octagon"; optional bool stop = 1; }
  22. wire { kotlin { emitDeclaredOptions = true emitAppliedOptions = true

    } } extend google.protobuf.MessageOptions { optional string documentation_url = 22200; } message Octagon { option (documentation_url) = "https://en.wikipedia.org/wiki/Octagon"; optional bool stop = 1; }
  23. // emitDeclaredOptions = true @Retention(AnnotationRetention.RUNTIME) @Target(AnnotationTarget.CLASS) public annotation class DocumentationUrlOption(

    public val value: String ) // emitAppliedOptions = true @DocumentationUrlOption("https://en.wikipedia.org/wiki/Octagon") public class Octagon( ... ) : Message<Octagon, Octagon.Builder>(ADAPTER, unknownFields) {}
  24. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { sourcePath { srcDir 'src/main/protos' } sourcePath { srcJar 'lib/pizza-protos.jar' } protoPath { srcJar 'com.example.pizza:pizza-protos:1.0.0' } ... }
  25. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { sourcePath { srcDir 'src/main/protos' include 'com/example/pizza/pizza_delivery.proto' include 'com/example/pizza/pizza.proto' } }
  26. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { sourcePath { srcDir 'src/main/protos' include 'com/example/pizza/pizza_delivery.proto' include 'com/example/pizza/pizza.proto' } }
  27. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { sourcePath { srcDir 'src/main/protos' include 'com/example/pizza/pizza_delivery.proto' include 'com/example/pizza/pizza.proto' } }
  28. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { protoPath { srcJar 'com.squareup.protos:all-protos' } sourcePath { srcDir 'src/main/protos' include 'com/example/pizza/pizza_delivery.proto' include 'com/example/pizza/pizza.proto' } }
  29. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { root 'com.example.store.Store' }
  30. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { prune 'com.example.store.Store' prune 'com.example.geo.Country' }
  31. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { kotlin { includes = ['com.example.pizza.*'] excludes = ['com.example.sales.*'] exclusive = false out "${buildDir}/custom" } }
  32. wire { ... kotlin { // Kotlin emits the matched

    types only. includes = [‘com.example.pizza.*'] exclusive = true } java { // Java gets everything else! } }
  33. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link

    Prune Generate wire { custom { schemaHandlerFactoryClass = "MyHandlerFactory" } } Schema Handler Coming soon™
  34. interface SchemaHandler { fun handle(schema: Schema, context: Context) interface Factory

    : Serializable { fun create(): SchemaHandler } } Coming soon™
  35. Common • common/money.proto • common/customer.proto // money.proto syntax = "proto3";

    package common; message Money { int64 amount = 1; Current currency = 2; } enum Currency { EUR = 0; GBP = 1; USD = 2; CAD = 3; } // customer.proto syntax = "proto3"; package common; message Customer { string token = 1; string name = 2; }
  36. Common • common/money.proto • common/customer.proto wire { sourcePath { srcDir("src/main/proto")

    } kotlin { } } sourceSets { val main by getting { resources.srcDir( project.file(“src/main/proto/") ) } }
  37. Common • common/money.proto • common/customer.proto wire { protoLibrary = true

    sourcePath { srcDir("src/main/proto") } kotlin { } }
  38. Common • common/money.proto • common/customer.proto wire { protoLibrary = true

    sourcePath { srcDir("src/main/proto") } kotlin { } } jar published internally to 'app.cash.common:common'
  39. Service Trades • trade/payment.proto syntax = "proto3"; package trade; import

    "common/customer.proto"; import "common/money.proto"; message PaymentRequest { string common.Money money = 1; string common.Customer recipient = 2; } message PaymentResponse {} service PaymentService { rpc Pay(PaymentRequest) returns (PaymentResponse); }
  40. Service Trades • trade/payment.proto wire { protoLibrary = true protoPath

    { srcJar("app.cash.common:common:<version>") } sourcePath { srcDir("src/main/proto") } kotlin { } }
  41. Service Trades • trade/payment.proto wire { protoLibrary = true protoPath

    { srcJar("app.cash.common:common:<version>") } sourcePath { srcDir("src/main/proto") } kotlin { } } dependencies { implementation("app.cash.common:common:<version>") }
  42. Service Trades • trade/payment.proto wire { protoLibrary = true protoPath

    { srcJar("app.cash.common:common:<version>") } sourcePath { srcDir("src/main/proto") } kotlin { } } dependencies { implementation("app.cash.common:common:<version>") } jar published internally to ‘app.cash.trade:trade'
  43. Mobile wire { sourcePath { srcJar("app.cash.common:common:<version>") } sourcePath { srcJar("app.cash.trade:trade:<version>")

    } sourcePath { srcJar("app.cash.balance:balance:<version>") } kotlin { android = true } }
  44. References • Protocol Bu ff ers documentations • https://developers.google.com/protocol-bu ff

    ers/ • Wire documentations • https://square.github.io/wire/ • Protogram • https://github.com/mattprecious/protogram