Real-time message handling and notifications with API Platform and Symfony

To meet the current demands for responsiveness and scalability in web applications and APIs, we will explore how to build a modern architecture for message handling using API Platform, Symfony Messenger and Mercure. Our goal is to establish a robust workflow that enables message processing and real time user notifications.

We will demonstrate how to orchestrate these technologies to deliver a seamless and immediate user experience, while ensuring high availability and optimal performance. This talk will give you a good foundation to address some of today’s performance and responsiveness requirements.

1. SEPTEMBER 19 -20, 2024 - LILLE, FRANCE & ONLINE Real-time message handling and notifications with API Platform and Symfony
2. About me Allison Guilhem ✔ Lead developer, Les-Tilleuls.coop ✔ Contributor to Symfony and API Platform @Alli_g83 Allison E. Guilhem
3. Real time notifications with Mercure via a use case scenario 01 02 Real time notifications: techniques and protocols
4. 01 - Real time notifications: techniques and protocols
5. HTTP Protocol ✔ Designed to transfer hypertext documents between a client and a server ✔ Request-response model ✔ HTTP1.0: A new TCP connection/request ✔ HTTP 1.1: It allows persistent connections / chunk transfer encoding, better caching etc ✔ HTTP2: Binary protocol / multiplexing / header compression / server push etc. ✔ HTTP3: It tackles the limitations of TCP (head-of-line blocking issue) / QUIC protocol / uses UDP/ connection more secure etc
6. Polling - long polling ✔ Straightforward ✔ Universal solution ✔ No persistent connections ✔ High latency and Increased bandwidth usage can be avoided with long polling ✔ Polling is quite inefficient for real time notifications ✔ Long polling: it keeps the connection open but it can put a lot of strain on the server / can be complex and have scalability issues
7. Server-Sent Events ✔ Simple to implement ✔ HTTP based ✔ JavaScript event listener to handle these updates ✔ Automatic Reconnection ✔ Text-Based Protocol ✔ Unidirectional Communication ✔ Scalability Issues, no builtin security mechanisms
8. WebSockets ✔ A lower-level protocol than HTTP ✔ A full-duplex, bidirectional communication channel ✔ WebSockets provide a lot of control but come with some drawbacks ✔ Low Latency ✔ Support Binary and Text Data ✔ They don’t fit as neatly with HTTP/2 and HTTP/3 ✔ Complexity ✔ Scalability Challenges ✔ Security Concerns: Authorization/Authentication, DoS attacks, tunnelling, sniffing attacks etc…
9. WebSub - Braid - PREP ✔ WebSub: lets servers subscribe to updates via webhooks. It’s limited to server-to-server communication only ✔ As Proposals and not fully implemented ✔ Braid: to handle real-time updates and synchronization of data across various clients. ✔ PREP: to provide a streamlined method for pushing updates from a server to clients, making it easier to implement real-time notifications and updates.
10. Mercure ✔ No extra setup needed ✔ Compatible with Any Server ✔ Automatic Reconnects and Syncs ✔ Secure with JWT ✔ High Performance ✔ GraphQL and Hypermedia-Friendly ✔ Built-in Encryption ✔ Backward Compatibility
11. Mercure Optional Hub
12. 02 -Real-time notifications with Mercure via a use case scenario
13. Use case with API Platform and Symfony ✔ Online casino platform ✔ Potential forbidden actions by users: fraud, connection from restricted countries etc ✔ System monitoring players activities: real time notifications are needed ✔ Admin dashboard is built with PHP Symfony and uses API Platform for their API
14. Setting up Mercure ✔ Via API Platform distribution ✔ Via Mercure Bundle using Symfony ✔ Composer require mercure ✔ Can install a Hub via docker
15. MercureBundle configuration ✔ Direct value / Provider / factory ✔ If none => Default LcobuccyFactory class ✔ If factory used (or default) ✔ Internally, FactoryTokenProvider will take into account the « subscribe » and « publish » claims
16. Mercure: debugging Tool Debugging tool
17. SuspiciousActivities endpoints
18. SuspiciousActivities endpoints
19. Publishing: Mercure options ✔ Topic: A list of topics/topics selectors for the update. the resource’s IRI is used as the default topic ✔ Data: This is the content of the update. The serialized form of the resource is used as the default ✔ ID: The SSE ID for this event. If not set, the Mercure Hub will generate an ID ✔ Type: The type of the SSE event. If not specified, this field is omitted ✔ Retry: The retry policy for SSE. If not defined, this field is omitted ✔ Normalization Context: The context to use for normalizing the update
20. Mercure Topics ✔ Static topic ✔ Dynamic Topic ✔ Via expression language in Symfony e.g/ placeholder/selectors ✔ Relation based topic ✔ Restrictive Updates via dynamic topics ✔ Allow subscribers to receive updates only for activities they’re allowed to see ✔ Useful in an online casino to notify specific users, such as security coordinators or managers, based on their roles or group affiliations ✔ It keeps sensitive information appropriately targeted ✔ S@=iri(object) ✔ @=iri(object.getCoordinator()) ~ "/? topic=" ~ escape(iri(object)) ✔ @=iri(object, ‘.UrlGeneratorInterface::ABS_PATH.) ✔ https://example.com/users/foo/{? topic} ✔ https://example.com/activities/* ✔ https://example.com/activities/{id} Examples It might evolve in v1
21. Mercure: restrictive update ✔ We only want to publish to a specific subscriber ✔ Publisher configuration as shown opposite ✔ Subscriber’s claim for their JWT token: https://mysite/api/users/3/ {?topic} e.g ✔ Modular and secure system
22. Mercure Topics Publisher Update with canonical and restrictive topics Subscriber With canonical topic Subscriber With restrictive update Subscriber Hub
23. Mercure: Publishing mechanism API Call Resource with ApiResource attribute and mercure option PublishMercureUpdatesListener Flush PostFlush - Verify options - Set ‘enable_async_upd ate: true’ if not specified -Build Update Class with iri, data, private, id, type, retry options as params Dispatch it through the bus with Messenger or directly publish to the hub Depending of ‘enable_async _update Via UpdateHandler synchronously or asynchronously Post HTTP request + JWT according to the claims Hub Publish to subscribers
24. Mercure: Publishing mechanism ✔ Enable_async_update: it can boost performance and it keeps the main processes of the application running smoothly, which is super important in a very high-traffic environment ✔ Scalability: the system can better manage traffic spikes, like during peak gaming events or multiple activities that need to be reported
25. Subscriber Publisher Hub SUBSCRIBE PUBLISH TOPICS Subscriber claims in JWT if private update Publisher claims in JWT NB: important to overlap, at least in part
26. Subscriber Publisher Hub SUBSCRIBE PUBLISH TOPICS Subscriber claims in JWT if private update Publisher claims in JWT IT WON’T WORK Or none or [] Or none or [] It won’t receive anything 401 received when the Hub tries to publish
27. And what about another approach?
28. Let’s trigger Messenger before Mercure Update ✔ Activities detection, especially fraud detection, often involves complex calculations or interactions with external systems ✔ Can be resource-intensive and could slow down the whole system
29. API Call Messenger Process Transport Handler - Heavy calculations - external Api calls etc - Construct Update w/ Update::class Hub Call Publish() method of the Hub::class Publish to subscribers
30. Publishing mechanism
31. Publishing mechanism
32. Could the messages be handled Concurrently?
33. Concurrent handling of messages
34. Concurrent handling of messages
35. Concurrent handling of messages
36. Concurrent handling of messages ✔ It would be possible to define how many processes should be batched and run concurrently before acknowledging them ✔ For better perf: use ext-parallel extension which depends on ZTS ✔ With FrankenPHP: ZTS is already enabled and parallel has recently become compatible with FrankenPHP ✔ In the child process, a container is cached ✔ Operations are performed concurrently for x number of processes ✔ Target ParallelMessageBus with BusNameStamp(‘parallel_bus’) ✔ php bin/console messenger:consume async_activities -p 20 ✔ Can be 5x times faster and less blocking if heavy operations are needed by some handlers This part might evolve
37. Subscriber Side
38. Discovering the hub ✔ ApiResource with mercure option: ✔ Link onto all the responses for that resource class ✔ Symfony: ✔ Discovery Helper Class
39. Subscribing
40. mercure() twig function Can be an array WithCredentials if private update
41. Set up the HubUrl with Topic(s) If lastEventId is defined, it adds: lastEventID as a query parameter If claims mentioned Authorization via cookie « mercureAuthorization » TokenFactory needed Payload JWT mercure() twig function: private
42. Last-Event-ID ✔ If connection drops: it tries to reconnect automatically ✔ Special header: Last-Event-ID ✔ Possible to define it in ApiResource Mercure option / or the hub will create it ✔ A way to get all the missed updates
43. Subscribing Authorization ✔ Via an authorization HTTP header ✔ The go-to method if the subscriber is not a web browser ✔ Via a cookie ✔ Preferred to name it mercureAuthorization ✔ Via an authorization URI query parameter ✔ Authorization=<JWS> ✔ Should have no store option and private option ✔ Only if nothing else works ✔ JWS should be short lived ✔ With Symfony: possible to set the authorization cookie directly in your code/ controller ✔ Authorization::class setCookie() ✔ Make sure to not define it twice!
44. Discover the Hub and subscribe efficiently ✔ @api-platform/mercure: A super handy eventSource wrapper with a mercure function ✔ Discover a mercureHub : fetch + new EventSource ✔ Very useful and it avoids multiple subscriptions with multiple connexions ✔ It manages multiple topics on a single connection ✔ It manages opening and closing of eventSource when updating the topics
45. API for active subscriptions ✔ It sends updates whenever a subscription is created or closed. This is crucial for real-time systems like online casinos, where monitoring active sessions is important ✔ /subscriptions: Lists all subscriptions ✔ /subscriptions/{topic}: Shows subscriptions for a specific topic ✔ /subscriptions/{topic}/{subscriber}: Gives details for a specific subscription ✔ The mercure claim of the JWT can also contain user-defined values under the payload key: a convenient way to share data related to one subscriber to other subscribers It might evolve in v1
46. Mercure ✔ A robust solution for real time communication ✔ Easy to deliver instant notifications ✔ A key protocol for meeting real-time communication demands ✔ Efficient across a variety of different platforms and environments ✔ Laravel community: some exciting developments are on the way for the server and client integration
47. Thank you! Thank you! Any questions?