Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ITT 2018 - Christopher Burnett - Types All the ...

ITT 2018 - Christopher Burnett - Types All the Way Down — gRPC and Go Infrastructure at Lyft

In this talk we'll discuss our experiences and lessons learned while introducing gRPC and Go into Lyft's ecosystem. We'll also cover how our Envoy proxy helps to bridge the type-safety gaps across polyglot systems and provides us with flexible and observable infrastructure solutions.

Istanbul Tech Talks

April 17, 2018
Tweet

More Decks by Istanbul Tech Talks

Other Decks in Programming

Transcript

  1. ABOUT US • Ride sharing - US and Canada •

    Interesting Scale and Challenges • Polyglot with ~500 engineers
  2. AGENDA • Why RPC? • Working with legacy systems •

    Disrupting workflows, nicely :) • What we’ve built
  3. NETWORKING A little about us… • HTTP & gRPC Libraries

    • Async Streaming Infrastructure • Envoy Proxy
  4. EVERY TEN YEARS… A furious bout of language and protocol

    design takes place and a new distributed computing paradigm is announced that is compliant with the latest programming model. - A Note On Distributed Computing, Waldo 1994
  5. TRIGGER WARNING • CORBA • Thrift • SOAP • WDDX

    • JSON-RPC • XML-RPC • Avro • HyperMedia • REST • MessagePack
  6. A LITTLE HISTORY Like any good story we begin with

    a PHP monolith… • Active decomposition efforts • 100s of Python Microservices - Flask HTTP/REST And to keep things interesting… • gRPC Core and Compositional Services
  7. DEFINING A CORE SERVICE • Organizational Primitives - User, Rides,

    Vehicles • Zero (Service) Dependencies - Databases, caches, etc • Highly Performant
  8. GRPC - THE BASICS • IDL (Interface Definition Language) Based

    • Protocol Buffer Wire Format • (Mostly) Full HTTP/2 Transport • Migration/Deprecation Options
  9. IDLS ARE PRETTY GREAT • Single Source of Truth -Primitive

    definitions • Code Generation -APIs, Clients, Servers, Data Models, Docs, Observability • Extensibility -Plugins for everything else
  10. IDL SERVICE DEFINITION package lyft.service.users.v1 service Users { rpc Update(UpdateRequest)

    UpdateResponse; } message UpdateRequest { uint64 id = 1; string name = 2; }
  11. IDL SERVICE DEFINITION — HTTP package lyft.service.users.v1 service Users {

    option (http.http_server_options).isHttpServer = true; rpc Update(UpdateRequest) returns UpdateResponse { // Override `path` for legacy URL support option (http.http_options).path = "/api/v1/users/:id"; option (http.http_options).method = "PUT"; } }
  12. TYPES ON THE WIRE • Simplified API I/O - Structs

    In, Structs Out • Safety - Big wins for dynamic languages • Transfer Cost - Improved latencies
  13. WHAT CAN MAKE THIS EASIER? • Incremental Adoption -Allow teams

    to opt-in to the new shiny things • Familiarity -Tooling that feels welcoming -Standardized framework patterns • Roll Forward -Wire format first, then the protocol and frameworks
  14. ENVOY PROXY - OVERVIEW • L4-L7 Proxy -Deployed at the

    edge or everywhere (sidecar) • Modern C++11 Codebase • Robust Metrics by Default • Highly Extensible -DB Monitoring, Protocol Upgrades
  15. Service Mesh ENVOY TOPOLOGY Internet Clients “Front” Envoy Legacy PHP

    Monolith Envoy Go Services Python Services Envoy Envoy MongoDB / DynamoDB Ratelimiting Stats / Tracing Discovery
  16. PROTOCOL BUFFER IDL (V3) • Service, Messages, and Types •

    Google blessed extensions - Well Known Types (WKTs), HTTP • Extension does become a necessity
  17. CURRENTLY… • Data Modeling • “Thick” Clients • Analytics Events

    • Validation • Service Generation • Documentation
  18. ODIE • DB Agnostic Object Document Mapper (ODM) -DynamoDB, Spanner,

    MongoDB • Protobuf IDL Schemas • Observability by Default -Distributed Tracing, Metrics, Logging • Highly Extensible -Decorator Pattern Extensions
  19. ODIE: IDLS MEET THE DATASTORE Type-Safe Repository Driver-Agnostic Client Expression

    Engine Decorator Middleware Database Driver IDL-Based Model Lifecycle Events MongoDB DynamoDB Spanner BoltDB
  20. ODIE: MODELS AS PROTOCOL BUFFERS message User { option (odie.mongo).enabled

    = true; string id = 1 [(odie.mongo).primary = true, (odie.type).object_id = true]; string name = 2 [(odie.mongo).name = "username"]; int64 date = 3 [(odie.type).datetime = true]; uint32 vers = 4 [(odie.locking).revision = true]; }
  21. ODIE: MODELS AS PROTOCOL BUFFERS type UserModel struct { Id

    bson.ObjectId `bson:"_id"` Name string `bson:"username"` Date time.Time Vers uint32 } func (pb *User) ToModel() *UserModel func (m *UserModel) ToProto() *User
  22. ODIE: TYPE-SAFE REPOSITORIES type UserRepo interface { Events() *Events Get(ctx

    context.Context, id bson.ObjectId) *GetBuilder Put(ctx context.Context, m *UserModel) *PutBuilder Delete(ctx context.Context) *DeleteBuilder Update(ctx context.Context) *UpdateBuilder Query(ctx context.Context) *QueryBuilder }
  23. EVENT INGEST • gRPC Streaming Analytics Ingestion • Compile Time

    Type Safety • Data Pipeline Agnostic • Tracing Abstraction
  24. PUSHER • gRPC Streaming API Proxy • Async Event Client

    Streams • Flexible API Infrastructure
  25. PUSH EVENTS message DriverLocation { option (event.push) = { route:

    "/v1/rides/:ride_id/driver-location", topics: ["pb.events.push.v1.rides.:ride_id.driver_location"] }; uint64 ride_id = 2 [(validate.rules).uint64.required = true]; }
  26. IT’S A PRETTY GOOD HAMMER message CompositeThing { option (odie.mongo).enabled

    = true; option (odie.spanner).enabled = true; common.EventBase event_base = 1 [(validate.rules).message.required = true]; option (event.push) = { route: "/v1/rides/:ride_id/driver-location", topics: ["pb.events.push.v1.rides.:ride_id.driver_location"] }; uint64 ride_id = 2 [(validate.rules).uint64.required = true]; }
  27. PROTOC-GEN-STAR (PG*) Code generation framework http://github.com/lyft/protoc-gen-star
 • AST of primitives

    • Simplifies code generation • Highly testable Package File Service Method Enum EnumValue Message OneOf Field
  28. PG*: WALK THE AST type Visitor interface { VisitPackage(Package) (v

    Visitor, err error) VisitFile(File) (v Visitor, err error) VisitMessage(Message) (v Visitor, err error) VisitEnum(Enum) (v Visitor, err error) VisitEnumValue(EnumValue) (v Visitor, err error) VisitField(Field) (v Visitor, err error) VisitOneOf(OneOf) (v Visitor, err error) VisitService(Service) (v Visitor, err error) VisitMethod(Method) (v Visitor, err error) }