Open standards for building event-driven applications in the cloud

AsyncAPI is an open-source specification for describing and documenting asynchronous APIs, similar to OpenAPI specification for documenting RESTful APIs. CloudEvents is a specification for event data in the cloud. Together, they enable developers to design, document, and test event-driven APIs and to easily share and consume event data across different cloud platforms and ecosystems.

In this session, we will explore the benefits of using AsyncAPI and CloudEvents in your tech stack, and how they can help you build asynchronous, event-driven applications that are well-documented and easy to maintain.

Transcript

1. Open standards for building event-driven applications in the cloud Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel
2. Introduction 01
3. An example event-driven architecture (EDA) An event-driven order processing system Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user Message Broker / Event Bus OrderReceived OrderReceived Payment Processed Payment Processed Items Shipped Items Shipped
4. In EDAs, you need to decide and communicate: ❏ The envelope of events ❏ Which service publishes or subscribes to what events?
5. TL;DR CloudEvents defines an envelope for events AsyncAPI helps to document event formats and which service publishes or subscribes to what events
6. CloudEvents 02
7. Why CloudEvents? Middleware Event source Event source Event source Event destination Event destination Event destination Different protocols: Kafka, MQTT, Pub/Sub, … Different formats: JSON, Protobuf, Avro, … Different event schemas Reading events and routing are difficult Kafka MQTT Google Pub/Sub
8. CloudEvents An open-source specification for describing events in a common way and SDKs to read and write CloudEvents Defines a minimal set of context attributes to route the event Data that is not intended for routing is placed within the data or data_base64 fields cloudevents.io
9. What does a CloudEvent look like? curl localhost:8080 -v -X POST \ -H "Content-Type: application/json" \ -H "ce-specversion: 1.0" \ -H "ce-type: com.mycompany.myapp.myservice.myevent" \ -H "ce-source: myservice/mysource" \ -H "ce-id: 1234-5678" \ -H "ce-time: 2023-01-02T12:34:56.789Z" \ -H "ce-subject: my-important-subject" \ -H "ce-extensionattr1: value" \ -H "ce-extensionattr2: 5" \ -d '{ "foo1": "bar1", "foo2": "bar2" }' Required Extension attributes Data Binary-mode: Good for receivers unaware of CloudEvents, as the metadata can be ignored Determines mode
10. What does a CloudEvent look like? curl localhost:8080 -v -X POST \ -H "Content-Type: application/cloudevents+json" \ -d '{ "specversion": "1.0", "type": "com.mycompany.myapp.myservice.myevent", "source": "myservice/mysource", "id": "1234-5678", "time": "2023-01-02T12:34:56.789Z", "subject": "my-important-subject", "datacontenttype": "application/json", "extensionattr1" : "value", "extensionattr2" : 5, "data": { "foo1": "bar1", "foo2": "bar2" } }' Required Extension attributes Data Structured-mode: Allows simple forwarding of the event across multiple routing hops Determines mode
11. What else CloudEvents provide? Event formats: JSON, Protobuf, Avro, XML (draft) Protocol bindings: HTTP, AMQP, Kafka, MQTT, NATS, WebSockets (draft) SDKs: Go, Javascript, Java, C#, Ruby, PHP, Python, Rust, Powershell Various CloudEvent libraries from vendors github.com/googleapis/google-cloudevents github.com/meteatamel/cloudevents-basics
12. However… Batch-mode not supported in all protocol/SDK combos e.g. Javascript SDK supports HTTP and Kafka batch mode but not MQTT batch mode Not all event formats are supported in each SDK e.g. Javascript SDK supports JSON but not Avro or Protobuf Not all protocol bindings are supported in each SDK e.g. Javascript SDK supports HTTP and Kafka but not AMQP, MQTT, NATS
13. Upcoming specs in CloudEvents Discovery API Subscriptions API Schema Registry
14. In EDAs, you need to decide and communicate: ✓ The envelope of events ❏ Which service publishes or subscribes to what events?
15. AsyncAPI 03
16. What is AsyncAPI? An open source specification to define an Async APIs, similar to what OpenAPI (aka Swagger) does for REST APIs Also tools to visualize and validate and generators to generate code from AsyncAPI definitions
17. Main concepts in AsyncAPI Application Server Channel Protocol Producer (Publisher) Consumer (Subscriber) Message Bindings for server, channel, operation, and message github.com/meteatamel/asyncapi-basics#concepts
18. OpenAPI AsyncAPI asyncapi.com/docs/tutorials/getting-started/coming-from-openapi vs.
19. AsyncAPI 2.6.0 vs 3.0.0 3.0.0 was released in December 2023 with breaking changes www.asyncapi.com/blog/release-notes-3.0.0 www.asyncapi.com/docs/migration/migrating-to-v3
20. What does an AsyncAPI definition look like? # The simplest AsyncAPI definition possible in 2.6.0 asyncapi: 2.6.0 info: title: Hello world application version: '0.1.0' channels: hello: publish: message: payload: type: string github.com/meteatamel/asyncapi-basics/tree/main/samples/hello-asyncapi
21. What does an AsyncAPI definition look like? # The simplest AsyncAPI definition possible in 3.0.0 asyncapi: 3.0.0 info: title: Hello world application version: 0.1.0 channels: hello: address: hello messages: publish.message: payload: type: string operations: hello.publish: action: receive channel: $ref: '#/channels/hello' messages: - $ref: '#/channels/hello/messages/publish.message'
22. Tools AsyncStudio Browser based tool to author and visualize and validate AsyncAPI files AsyncAPI CLI CLI based tool to work with AsyncAPI files AsyncAPI Generator Code generators for various languages & frameworks github.com/meteatamel/asyncapi-basics/tree/main/samples/quickstart
23. Publish/subscribe semantics in AsyncAPI 2.6.0 In a channel, you can have publish and subscribe operations AsyncAPI Term WebSocket Term From Server Perspective From User Perspective Publish Send The server receives messages The user publishes messages to server Subscribe Receive The server sends messages The user subscribes to receive messages from server github.com/meteatamel/asyncapi-basics/tree/main/samples/account-email-services Most useful
24. Send/receive semantics in AsyncAPI 3.0.0 In AsyncAPI 3.0.0, channels don’t have operations. Instead, there’s a separate operations section In operation, you can have send and receive actions from server’s perspective github.com/meteatamel/asyncapi-basics/tree/main/samples/account-email-services
25. In EDAs, you need to decide and communicate: ✓ The envelope of events ✓ Which service publishes or subscribes to what events?
26. CloudEvents+AsyncAPI Google Pub/Sub+AsyncAPI 04
27. Can you use CloudEvents and AsyncAPI together? Yes! github.com/meteatamel/asyncapi-basics/tree/main/samples/account-service-cloudevents
28. Can you use Google Pub/Sub and AsyncAPI together? Yes! github.com/meteatamel/asyncapi-basics/tree/main/samples/google-pubsub
29. Thank you! Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel cloudevents.io github.com/meteatamel/cloudevents-basics asyncapi.com github.com/meteatamel/asyncapi-basics Feedback? bit.ly/atamel