Practical insight into modulithic architecture

In software architecture, it is always about finding the best solution for your use case and your application in your specific context. Based on that, you have the challenge to find the right balance between flexibility and simplicity. We often must decide between a monolithic system and a microservice architecture. For a long time, the monolithic architecture has been the way to go. Then, microservices came up and most architectures followed way. Today I would like to present an architecture that attempts to combine the advantages of both paradigms.

Using a concrete example project, I will discuss the advantages and challenges of a modular architecture. I will present a possible approach to the appropriate structuring of Symfony applications using modulithic architecture and show how it can help to map modern software development requirements.

Transcript

1. Practical insight into modulithic architecture Max Beckers 𝕏 @maxbbeckers Elbphilharmonie Hamburg
2. Contact: 𝕏 @maxbbeckers beckers.maximilian@gmail.com linkedin.com/in/maximilianbeckers Max Beckers Solutions Architect • Working for / • Living near Kiel, Germany • Symfony Experience since 2012
3. Software architecture is always about finding the best solution for your use case and your application in your specific context. The challenge is to find the right balance between simplicity and flexibility. 𝕏 @maxbbeckers
4. Simplicity in focus Monoliths 𝕏 @maxbbeckers
5. Focusing on flexibility Microservices 𝕏 @maxbbeckers
6. Finding its place between Monoliths and Microservice Modulithic Architecture Modular Monolith 𝕏 @maxbbeckers
7. 𝕏 @maxbbeckers Monolith – Modulith – Microservice – Overview Microservices Modulith Monolith Multiple Both possible Single Codebase Multiple Single Single Releases Multiple Single Single Deployments Domain / Business centered Domain / Business centered Technical design Design Principles Orchestration & Monitoring Releases / Coordination Understanding Code Complexity Network overhead Direct calls Direct calls Performance Services + Integration Modules + Integration Whole codebase Testing
8. 𝕏 @maxbbeckers Monolith – Modulith - Microservice Microservices Modulith Monolith ❌ ✔ ✔ Simplicity ✔ ❌* ❌ Scalability ✔ ✔ ❌ Low Coupling ✔* ✔* ❌* Maintainability ❌ ✔ ✔ Single point of failure ✔ ❌* ❌ Free in technology usage ❌ ✔ ✔ Transaction Handling
9. Let's dive into the example … 𝕏 @maxbbeckers
10. 𝕏 @maxbbeckers
11. 𝕏 @maxbbeckers
12. 𝕏 @maxbbeckers Architecture Overview API GATE WAY Payment Processing Order Management Customer Management SEPA Mandate Payment Provider
13. 𝕏 @maxbbeckers Symfony Application Overview Use-Case based Structure instead of technical namespaces at the first level.
14. 𝕏 @maxbbeckers Generate Code from the API #!/usr/bin/env bash openapi-generator-cli generate \ -i ./public-contract-definition.yaml \ -g php \ -o ./generated \ --additional-properties=\ invokerPackage=Worldline\\Payments\\Api\\V2,\ modelPackage=Models,\ artifactVersion=2.279.0,\ composerPackageName=worldline/payment-api,\ licenseName=MIT
15. 𝕏 @maxbbeckers Composer part "repositories": [ { "type": "path", "url": "api-spec/generated", "options": { "symlink": true } } ], $ composer require worldline/payment-api:@dev ./composer.json has been updated Running composer update worldline/payment-api Loading composer repositories with package information Updating dependencies
16. 𝕏 @maxbbeckers API Gateway "Internal" namespace denotes module internal functionality – cannot be accessed outside the module context.
17. 𝕏 @maxbbeckers Modulith internal APIs - Objects Modules have data structures defined by DTOs
18. 𝕏 @maxbbeckers Modulith internal APIs - Services Services in a module are called using interfaces. In a Microservices context, this could be the OpenAPI Spec.
19. 𝕏 @maxbbeckers Modulith internal APIs - Messages Modules can consume messages defined in Message. namespace App\Order\Message; use App\Order\Dto\OrderDto; class PersistOrderEvent { public function __construct( private readonly OrderDto $orderDto ) { } public function getOrderDto(): OrderDto { return $this->orderDto; } }
20. 𝕏 @maxbbeckers Modulith internal APIs - Overview • DTOs – Data Structure for the API of the Module • Interfaces – Endpoints for the API of the Module • Messages – Async Communication API of the Module • Exceptions – Error Responses of the Module
21. 𝕏 @maxbbeckers #[Route(name: "app_v2_payments_")] class PaymentsController extends AbstractController { public function __construct( private readonly PaymentServiceInterface $paymentService, private readonly PaymentMapper $paymentMapper ) {} #[Route('/v2/{merchantId}/payments', name: 'create_payment', methods: ['POST'])] public function create(Request $request, string $merchantId): JsonResponse { /** @var CreatePaymentRequest $createPaymentRequest */ $createPaymentRequest = ObjectSerializer::deserialize($request->getContent(), CreatePaymentRequest::class); $paymentRequest = $this->paymentMapper->mapCreatePaymentRequestToPaymentDTO($createPaymentRequest, $merchantId); $paymentResponse = $this->paymentService->pay($paymentRequest); $response = $this->paymentMapper->mapPaymentDTOToPaymentResponse($paymentResponse); return $this->json(ObjectSerializer::sanitizeForSerialization($response), 201); } } Payments Controller
22. 𝕏 @maxbbeckers #[Route(name: "app_v2_payments_")] class PaymentsController extends AbstractController { public function __construct( private readonly PaymentServiceInterface $paymentService, private readonly PaymentMapper $paymentMapper ) {} #[Route('/v2/{merchantId}/payments', name: 'create_payment', methods: ['POST'])] public function create(Request $request, string $merchantId): JsonResponse { /** @var CreatePaymentRequest $createPaymentRequest */ $createPaymentRequest = ObjectSerializer::deserialize($request->getContent(), CreatePaymentRequest::class); $paymentRequest = $this->paymentMapper->mapCreatePaymentRequestToPaymentDTO($createPaymentRequest, $merchantId); $paymentResponse = $this->paymentService->pay($paymentRequest); $response = $this->paymentMapper->mapPaymentDTOToPaymentResponse($paymentResponse); return $this->json(ObjectSerializer::sanitizeForSerialization($response), 201); } } Payments Controller
23. 𝕏 @maxbbeckers #[Route(name: "app_v2_payments_")] class PaymentsController extends AbstractController { public function __construct( private readonly PaymentServiceInterface $paymentService, private readonly PaymentMapper $paymentMapper ) {} #[Route('/v2/{merchantId}/payments', name: 'create_payment', methods: ['POST'])] public function create(Request $request, string $merchantId): JsonResponse { /** @var CreatePaymentRequest $createPaymentRequest */ $createPaymentRequest = ObjectSerializer::deserialize($request->getContent(), CreatePaymentRequest::class); $paymentRequest = $this->paymentMapper->mapCreatePaymentRequestToPaymentDTO($createPaymentRequest, $merchantId); $paymentResponse = $this->paymentService->pay($paymentRequest); $response = $this->paymentMapper->mapPaymentDTOToPaymentResponse($paymentResponse); return $this->json(ObjectSerializer::sanitizeForSerialization($response), 201); } } Payments Controller
24. 𝕏 @maxbbeckers Persistence & Data
25. 𝕏 @maxbbeckers • Each module has its own tables • Modules are not integrated at the database level (SELECTS / JOINS using tables of other modules are not allowed) 4 ways of data isolation: • Different type of persistence (e.g. MySQL, Postgres, Redis, MongoDB, …) • Separate database for each module • Separate schema for each module (used in the example) • Separate tables in the same schema (I would not recommend) Persistence & Data Rules
26. 𝕏 @maxbbeckers Database setup API GATE WAY Payment Processing Order Management Customer Management SEPA Mandate Payment Order Customer Mandate
27. 𝕏 @maxbbeckers Entities <?php namespace App\Mandate\Internal\Entity; use Doctrine\ORM\Mapping as ORM; use Symfony\Bridge\Doctrine\Types\UuidType; use Symfony\Component\Uid\Uuid; #[ORM\Entity] class Customer { #[ORM\Id] #[ORM\Column(type: UuidType::NAME, unique: true)] private Uuid $id; #[ORM\Column(length: 64)] private string $merchantId; #[ORM\Column(length: 36)] private string $customerReference; #[ORM\Column(length: 34)] private string $iban; #[ORM\OneToMany(targetEntity: Mandate::class, mappedBy: 'customer')] private array $mandates;
28. 𝕏 @maxbbeckers Database config .env CUSTOMER_DATABASE_URL="postgresql://customer:password@127.0.0.1:5432/customer?serverVersion=16&charset=utf8" MANDATE_DATABASE_URL="postgresql://mandate:password@127.0.0.1:5432/mandate?serverVersion=16&charset=utf8" ORDER_DATABASE_URL="postgresql://order:password@127.0.0.1:5432/order?serverVersion=16&charset=utf8" PAYMENT_DATABASE_URL="postgresql://payment:password@127.0.0.1:5432/payment?serverVersion=16&charset=utf8" Can be different for each module Can be different for each module
29. 𝕏 @maxbbeckers Database config doctrine: dbal: connections: customer: url: '%env(resolve:CUSTOMER_DATABASE_URL)%' mandate: url: '%env(resolve:MANDATE_DATABASE_URL)%' order: url: '%env(resolve:ORDER_DATABASE_URL)%' payment: url: '%env(resolve:PAYMENT_DATABASE_URL)%' default_connection: payment orm: entity_managers: customer: connection: customer mappings: Customer: type: attribute is_bundle: false dir: '%kernel.project_dir%/src/Customer/Internal/Entity' prefix: 'App\Customer\Internal\Entity' alias: Customer
30. 𝕏 @maxbbeckers Doctrine Migrations #config/packages/migrations/migrations_customer.yaml em: customer migrations_paths: App\Migrations\Customer: 'migrations/Customer' table_storage: table_name: migration_versions $ php bin/console doctrine:migrations:diff --em=customer --configuration=config/packages/migrations/migrations_customer.yaml Generated new migration class to "migrations/Customer/Version20240502123126.php" To run just this migration for testing purposes, you can use migrations:execute --up 'App\\Migrations\\Customer\\Version20240502123126' To revert the migration you can use migrations:execute --down 'App\\Migrations\\Customer\\Version20240502123126'
31. 𝕏 @maxbbeckers What about queues?
32. 𝕏 @maxbbeckers Messenger setup API GATE WAY Payment Processing Order Management Customer Management SEPA Mandate Payment Order Customer
33. 𝕏 @maxbbeckers Queue config .env CUSTOMER_MESSENGER_TRANSPORT_DSN=amqp://customer:password@localhost:5672/customer/messages ORDER_MESSENGER_TRANSPORT_DSN=amqp://order:password@localhost:5672/order/messages PAYMENT_MESSENGER_TRANSPORT_DSN=amqp://payment:password@localhost:5672/payment/messages framework: messenger: transports: customer: dsn: '%env(CUSTOMER_MESSENGER_TRANSPORT_DSN)%' order: dsn: '%env(ORDER_MESSENGER_TRANSPORT_DSN)%' payment: dsn: '%env(PAYMENT_MESSENGER_TRANSPORT_DSN)%' routing: 'App\Customer\Message\PersistCustomerEvent': customer 'App\Order\Message\PersistOrderEvent': order 'App\Payment\Internal\Message\PersistPaymentEvent': payment
34. 𝕏 @maxbbeckers Scaling Consumers with Supervisor $ sudo apt-get install supervisor ;/etc/supervisor/conf.d/messenger-worker-payment.conf [program:messenger-consume] command=php /home/www/bin/console messenger:consume payment --time-limit=3600 user=ubuntu numprocs=2 startsecs=0 autostart=true autorestart=true startretries=10 process_name=%(program_name)s_%(process_num)02d
35. 𝕏 @maxbbeckers Flexibility API GATE WAY Payment Processing Order Management Customer Management SEPA Mandate
36. 𝕏 @maxbbeckers Add a new module API GATE WAY Payment Processing Order Management Customer Management SEPA Mandate Invoicing
37. 𝕏 @maxbbeckers Modulith is an architectural alternative between Monolith and Microservices • Simplicity & Maintainability • Low coupling & clear boundaries • Less complexity Monolith: Understanding code, Testing, Clear boundaries Microservices: Easier to configure, no network overhead The Complexity of the business code itself stays of course 😉 Summary
38. 𝕏 @maxbbeckers Don’t start with microservices “You shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.” Martin Fowler S
39. 𝕏 @maxbbeckers Source: https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90
40. Contact: 𝕏 Thanks for your attention! @maxbbeckers beckers.maximilian@gmail.com linkedin.com/in/maximilianbeckers