Distributed Transactions with Symfony

In microservice or event-driven architectures you might find yourself having to solve issues which are simple when working with monolithic applications. One of these challenges is distributed transactions, i.e. performing an action across multiple services can be safely reversed when something fails along the call chain. The Saga pattern is an established approach to make this happen. Let's have a look at what it is, how we can implement it in a modern Symfony application and when we might want to avoid it.

Transcript

1. DISTRIBUTED TRANSACTIONS WITH SYMFONY A LOOK AT THE SAGA PATTERN 04.04.2025
2. Next Kraftwerke GmbH Lichtstr. 43g 50825 Cologne Germany +49 221 – 820085-0 info@next-kraftwerke.de Denis Brumann Solutions Architect WHO AM I brumann@next-kraftwerke.de
3. Transaction Distributed Transactions with Symfony DATABASE TRANSACTIONS 4 TRANSACTION HANDLING IN SYMFONY User Address 04.04.2025
4. Transaction User Address Distributed Transactions with Symfony ADDRESS #[Entity] #[Table(name: ‘app_addresses')] class Address { #[Id] #[GeneratedValue] #[Column(type: 'integer', options: ['unsigned' => true])] private ?int $id; #[Column(type: 'string', options: ['length' => 255])] private string $street; #[Column(type: 'string', options: ['length' => 255])] private string $city; #[Column(type: 'string', options: ['length' => 31])] private string $zipCode; #[Column(type: 'string', options: ['length' => 255])] private string $country; } 5 ENTITY 04.04.2025
5. Distributed Transactions with Symfony USER #[Entity(repositoryClass: UserRepository::class)] #[Table(name: ‘app_users')] class User { #[Id] #[GeneratedValue(strategy: "AUTO")] #[Column(type: 'integer', options: ['unsigned' => true])] private ?int $id; #[Column(type: 'string', options: ['length' => 255])] private string $displayName; #[Column(type: 'string', options: ['length' => 255])] private string $email; #[OneToOne(targetEntity: Address::class, cascade: ['persist', 'remove'], orphanRemoval: true)] private Address $address; } 04.04.2025 6 ENTITY WITH ASSOCIATED ADDRESS
6. Distributed Transactions with Symfony USERCONTROLLER #[AsController] final readonly class UserController { #[Route(path: '/', name: 'user_create', methods: ['GET', 'POST'])] public function create(Request $request): Response { $userForm = $this->formFactory->create(UserType::class); $userForm->handleRequest($request); if ($userForm->isSubmitted() && $userForm->isValid()) { $this->entityManager->persist($userForm->getData()); $this->entityManager->flush(); $this->entityManager->clear(); return new RedirectResponse($this->urlGenerator->generate('user_list')); } return new Response($this->renderer->render('users/create.html.twig', [ 'user_form' => $userForm->createView(), ])); } } 7 WHERE THE MAGIC HAPPENS 04.04.2025
7. Distributed Transactions with Symfony DISTRIBUTED TRANSACTIONS A quick example Persisting a user with an address. We have two entities (User & Address), a Symfony Form, Twig Template and a UserController containing the logic for persisting the User with its address. What will happen… ▪ Doctrine throws an Exception when cascade is missing on the OneToOne-Attribute/Annotation ▪ Once the problem is fixed, everything works as expected and Doctrine wraps both inserts in a transaction without you needing to declare it ▪ You can persist both entities individually, without adding the attribute, Doctrine will still wrap it in a transaction for you ▪ You want to be explicit or ensure something happens on rollback and manually add a beginTransaction()/ transactional() call 8 SYMFONY + DOCTRINE 04.04.2025
8. Distributed Transactions with Symfony PROFILER 9 WHAT IS HAPPENING UNDER THE HOOD? Start Transaction End Transaction Insert address Insert user 04.04.2025
9. Saga Service Distributed Transactions with Symfony THE SAGA PATTERN 10 MANAGE TRANSACTIONS AS A SERIES OF LOCAL TRANSACTIONS Service Service Message Message 04.04.2025
10. Distributed Transactions with Symfony TRANSACTIONS ACROSS MICRO SERVICES 11 Source: https://learn.microsoft.com/en-us/dotnet/architecture/cloud-native/distributed-data 04.04.2025
11. Distributed Transactions with Symfony COMPENSATING TRANSACTIONS 12 HANDLING ERRORS IN THE TRANSACTION CHAIN Source: https://learn.microsoft.com/en-us/dotnet/architecture/cloud-native/distributed-data 04.04.2025
12. Distributed Transactions with Symfony Compensable transaction Can be undone using a compensating transaction having rolling back the changes of the compensable transaction.. Ref. “Create Pending Order” — “Cancel Order”, “Validate Payment” — “Cancel Payment” 13 TYPICAL TRANSACTION TYPES Pivot transaction Serve as “point of no return” in the saga. Any preceding compensable transaction no longer needs to be compensated when something fails. It usually serves the following purposes: ▪ Irreversible transaction which is neither compensable or retryable ▪ Boundary between reversible and committed, meaning it is either the last compensable transaction or the first retry-able operation in the saga. Retry-able transaction They are idempotent instead of reversible. In other words, instead of rolling back we can try to attempt the same action again without having to fear data corruption. Noticeably, they are still considered part of the saga, meaning for the saga to be completed successfully these steps need to be performed. 04.04.2025
13. Saga Coordinator Distributed Transactions with Symfony ORCHESTRATION ▪ Command-driven ▪ Controlled by a central service (Coordinator/Orchestrator) ▪ Coordinator controls the process flow, which usually makes error handling and debugging easier ▪ Allows for complex workflows 14 MANAGE TRANSACTIONS AS A SERIES OF LOCAL TRANSACTIONS Service Other Service 04.04.2025
14. Distributed Transactions with Symfony ORCHESTRATION Benefits ▪ Works well for complex workflows requiring coordination; arguably easier to extend than choreography ▪ Logic is coordinated transparently in an orchestrating service, reducing complexity and minimizing some risks inherent in distributed systems ▪ Keeping most complexity in the orchestrator reduces it in the other saga participants; potentially reduces infrastructure complexity as well 15 PROS & CONS Drawbacks ▪ Implementing coordination logic can make the orchestrator more complex ▪ Orchestrator can become the single point of failure for the saga 04.04.2025
15. Saga Distributed Transactions with Symfony CHOREOGRAPHY ▪ Choreography does not rely on a central coordinating service. ▪ Decentralized architecture (no single point of failure) ▪ Participants only know their role and are working independently from each other ▪ Event-based approach allows for easier decoupling of steps 16 USING A CENTRALIZED CONTROLLER Message Broker Service Other Service 04.04.2025
16. Distributed Transactions with Symfony CHOREOGRAPHY Benefits ▪ Works best for multiple, concurrent tasks without coordination between them ▪ No coordinating service required; (existing) message broker can be reused for multiple sagas ▪ Distributed responsibilities reduces risk of single point of failure 17 PROS & CONS Drawbacks ▪ Indirection makes it hard to track progress or even source of saga transaction, making it harder to understand what is happening together ▪ Having many saga participants can make it difficult to understand what is happening and can introduce new risks, like cyclic dependencies ▪ Testing & debugging quickly becomes difficult due to infrastructure complexity 04.04.2025
17. Distributed Transactions with Symfony COMPARISON 18 ORCHESTRATION VS. CHOREOGRAPHY Source: https://camunda.com/blog/2023/02/orchestration-vs-choreography/ 04.04.2025
18. Distributed Transactions with Symfony WHEN TO USE WHICH Other Alternatives ▪ Neither ▪ Maybe Saga should be a database transaction instead? ▪ Mixed Approach ▪ Combine orchestration & choreography ▪ (Routing Slip integration pattern) ▪ Queues actions, but does not cover “rollback” 19 ORCHESTRATION VS. CHOREOGRAPHY Source: https://medium.com/ultimate-systems-design-and-building/choreography-vs-orchestration-in-microservices-which-saga-strategy-should-you-choose-be0bb700a1d2 04.04.2025
19. Distributed Transactions with Symfony phpsagas/orchestrator Provides a nice fluent interface for defining a Saga using established terminology. Split into multiple packages and generally looks well designed. The orchestrator has an exceptional documentation in the README, which is valuable even if the package is inactive and likely not an advisable dependency. Last updated 4 years ago and maintainer’s GitHub account seems to be inactive. 20 SAGA IN PHP OPEN SOURCE PACKAGES brzuchal/saga Saga implementation using a dedicated saga repository for managing the lifecycle and attributes for setting up the saga orchestration, but seems to be tied to Symfony 6.. Symfony integration is available in a dedicated repository There is an order example in another repository showing it in action. Last updated in December 2023. php-service-bus/service-bus A service bus implementation for PHP, which is not (just) tailored for Sagas; provides building blocks for implementing sagas. Tied to Symfony 6 and development seems to have stopped rather abruptly with incomplete docs changes. You can find an example for Sagas in the 5.0-branch: / v5.0/.documentation/sagas.md Last updated 3 years ago. 04.04.2025
20. Distributed Transactions with Symfony WHAT COULD HAVE BEEN Example: Multi-step hotel booking 21 PHP-SAGAS/ORCHESTRATOR 04.04.2025
21. Distributed Transactions with Symfony SAGAS WITH SYMFONY Symfony Messenger ▪ Issue commands to a message broker ▪ Worker listens to events for success/failure to either issue the next transaction or compensating transactions in reverse ▪ Retryable transactions can theoretically be handled using messenger retries, but are outside of the control of the coordinator ▪ If this is not suitable, consider a custom Stamp on the command and maybe the Failure-event to allow coordinator to decide on appropriate retry behavior. 22 SENDING TRANSACTIONS Symfony HTTP Client ▪ Perform requests to participants to perform transactions ▪ In case of an error response, initiate compensating transactions ▪ Bonus: Handling infrastructure issues (timeouts) usually is easier than with messages ▪ Symfony’s HttpClient-component natively supports retry with the RetryableHttpClient 04.04.2025
22. Distributed Transactions with Symfony SAGAS WITH SYMFONY Symfony Lock ▪ Compensable transactions might want to implement a lock (using an appropriate store) to prevent anomalies from conflicting/concurrent changes ▪ While it might be tempting to share the lock between coordinator and participant(s) it can lead to issues when they become more independent ▪ When a lock can not be acquired the participant should report the error to the coordinator instead of having the coordinator check whether a lock is available for the participant 23 PREVENTING DATA ANOMALIES Doctrine ▪ Doctrine provides support for locking ▪ Optimistic locking requires a version field on your entity, which then ensures updates happen in the right order ▪ Since we are not sharing a database, this is mostly relevant on the coordinating side, when the individual steps produce an update on the “main” object ▪ If all you need, is making sure the saga steps happen in the right order, you might want to consider using the Workflow-component instead 04.04.2025
23. Distributed Transactions with Symfony ADVANCED ORCHESTRATION WITH SYMFONY 24 WORKFLOW-COMPONENT framework: workflows: order_saga: type: 'state_machine' supports: - App\Entity\OrderSaga initial_marking: pending places: [ pending, paymentValidated, inventoryUpdated, shipmentSourced, invoiceSent, completed ] transitions: validatePayment: { from: pending, to: paymentValidated } updateInventory: { from: paymentValidated, to: inventoryUpdated } sourceShipment: { from: inventoryUpdated, to: shipmentSourced } sendInvoice: { from: shipmentSourced, to: invoiceSent } complete: { from: invoiceSent, to: completed } cancel: from: [ paymentValidated, inventoryUpdated, shipmentSourced, invoiceSent ] to: pending 04.04.2025
24. Distributed Transactions with Symfony ADVANCED ORCHESTRATION WITH SYMFONY 25 WORKFLOW-COMPONENT 04.04.2025
25. Distributed Transactions with Symfony MORE ADVANCED ORCHESTRATION “Open source durable execution system. Write code that’s fault- tolerant, durable, and simple.” ▪ Provides SDKs for multiple languages, including PHP ▪ Web UI for managing workflows ▪ Failure detection and mitigation ▪ Open Source (MIT license) ▪ Self-hosted & Cloud 26 TEMPORAL.IO Source: https://temporal.io/ 04.04.2025
26. Distributed Transactions with Symfony SAGA LOG Monolog ▪ Writing Saga-changes using a dedicated SagaLogger can be helpful to (manually) resolve data inconsistencies ▪ What happens, if a compensating transaction fails? Can you retry the step? ▪ Beware, different services might configure their logs differently. Ideally, the coordinator receives all relevant info even from (failing) remote commands to allow for uniform, consistent logging. ▪ If you store your saga lifecycle in a dedicated entity, you could look into audit logging, e.g. using something like damienharper/auditor-bundle 27 HANDLING FAILURES 04.04.2025
27. Distributed Transactions with Symfony Planning Saga introduces complexity. Do you really need it or are there better alternatives for you? If you go with Saga, consider pros and cons of orchestration vs. choreography. Discussing business requirements and tradeoffs with stakeholders is advisable. 28 SUMMARY Implementation Existing libraries providing saga-capabilities in PHP/ Symfony might be outdated, but can still give inspiration for a more generalized approach in your codebase. Saga orchestration with Symfony can be achieved by utilizing existing components Workflows are particularly useful for deliberately designing saga- orchestration and even provide a visual representation. Third party tools Despite the lack of mature libraries in the PHP ecosystem, you can use other resources for managing your saga lifecycle. There are dedicated services like temporal.io or serverless workflow orchestrations by cloud providers, e.g. AWS Step Functions. Details The devil is in the details. Thorough end-to-end testing of success and failure scenarios is key to prevent data inconsistency Locking can further prevent issues from concurrent or out of order transactions.. A dedicated Saga log can help you recover data, when something did go wrong. 04.04.2025
28. Next Kraftwerke GmbH Lichtstr. 43g 50825 Cologne Germany +49 221 – 820085-0 info@next-kraftwerke.de DANKE