Integration Platform: Basic example and decisions

This is a presentation to expose an example on my personal blog.

https://www.duvanvilladiego.dev/blog/understanding-software-architecture-basics-importance-and-styles

Transcript

1. Integration Platform Duvan Andres Villadiego Hoyos
2. Problem
3. Application need an Integration Platform We are designing an integration platform for an application that needs to interact with multiple external services. Since we cannot predict how long external services will take to process each request, the system must support long-running operations. Because this platform is a complement to the main application, integration processing must not directly affect the performance of the core system.
4. Requirements and Constraints
5. Business Requirements • The system must integrate with third-party services such as payment providers, shipping services, and notification platforms. • New integrations should be added without affecting the rest of the system. • Integration failures must not stop the core business operations.
6. Functional Requirements • The module must send requests to external APIs. • The module must receive and process responses from external systems. • The system must handle different integration types (REST APIs, webhooks, messaging systems). • The module must retry failed requests when external services are temporarily unavailable. • The module must log all integration requests and responses.
7. Technical Constraints • The system must use REST APIs to communicate with external services. • The system must support asynchronous processing for long-running integrations. • All communication with external services must use HTTPS. • The system must implement timeout and retry mechanisms.
8. Proposals
9. 1. Synchronous Integration Service A dedicated microservice exposes an API that allows other services to request integrations. The request specifies: • The type of integration (payment, shipping, notifications). • The provider to use. The integration service communicates with the external provider and returns the result. Advantages: • Simple architecture. • Easy to implement. • Direct communication. Limitations: • External service latency affects response time. • Failures can impact the caller service.
11. 2. Asynchronous Integration Processor Requests are stored in a database and processed by a worker using a state machine pattern that tracks the lifecycle of each integration. Example lifecycle: Pending → Processing → Completed → Failed Advantages: • More resilient. • Better for long-running integrations. • Allows retries and failure recovery. Limitations: • Higher architectural complexity. • Requires state management. • Highly database queries that can saturate the database core.
13. Decision - Asynchronous Integration Processor Why? Based on the following requirement: • The system must support asynchronous processing for long-running integrations. We can assume that many integrations may take a significant amount of time to complete. Because of this, a solution that does not block the core system while waiting for external services is a better architectural choice. Main Pros: • Higher resilience when interacting with external systems. • Better support for long-running operations. • Prevents blocking core system processes. Main Const: • Higher initial complexity. • Potential database saturation if requests are not managed properly.