Service orchestration patterns

Once you have a fleet of services, you need to decide how to organize and get them to cooperate for a common goal. Should they communicate with direct calls? Or should they communicate indirectly in a loosely coupled way via events? Maybe a central orchestrator should direct the communication? What do you do when a service fails? When to use a simple retry or a more sophisticated Saga pattern? In this talk, we'll look at some orchestration patterns and techniques and to get your services to cooperate in a resilient way.

Transcript

1. Service orchestration patterns Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel
2. Service orchestration
3. How do you organize a group of microservices to cooperate towards a common goal?
4. Option 1: Direct service-to-service calls Services calling each other directly Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user
5. Direct service-to-service calls Pros ➕ Easy to implement: Services simply call each other Cons ➖ Too much coupling ➖ Each service can be a single point of failure ➖ Each service needs its own error / retry / timeout logic ➖ Who ensures the whole transaction is successful? (hint: saga pattern)
6. Option 2: Indirect via events (choreography) Event-driven services Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user Message Broker Google Cloud: Pub/Sub, Eventarc AWS: SQS, SNS, EventBridge Azure: Event Grid, Event Hubs, Service Bus Other: Kafka, Pulsar, Solace PubSub+, RabbitMQ, NATS...
7. Indirect via events Pros ➕ Services are loosely coupled ➕ Services can be changed/scaled independently ➕ No single point of failure ➕ Events are useful to extend the system Cons ➖ Difficult to monitor ➖ Errors / retries / timeouts are hard ➖ The business flow is not captured explicitly ➖ Who ensures the whole transaction is successful?
8. Imagine a more complex scenario
9. Option 3: A central orchestrator Orchestrated services Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user Orchestrator Google Cloud: Workflows, Cloud Composer AWS: Step Functions Azure: Logic Apps Other: CNCF Serverless Workflow, Apache Airflow, Camel, Camunda…
10. A central orchestrator Pros ➕ Business flow captured centrally, source controlled, versioned etc. ➕ Each step can be monitored ➕ Errors / retries / timeouts can be centralized ➕ Services are still independent Cons ➖ A new orchestrator service to learn and maintain ➖ Orchestrator could be a single point of failure ➖ Loss of eventing flexibility ➖ How do you compensate for failed steps? (hint: saga pattern)
11. Patterns and best practices
12. Make a conscious choice Indirect Events ➔ Services are not closely related ➔ Services are not executed in parallel or in no certain order ➔ Services can exist in different bounded contexts Central Orchestration ➔ Services are closely related ➔ Services are usually deployed and executed in the same order ➔ Can you describe the architecture in a flow chart? Direct Calls ➔ A simple architecture with a handful of services
13. Event-driven orchestration github.com/GoogleCloudPlatform/eventarc-samples/tree/main/processing-pipelines/image-v3
14. Handle errors with retries and saga pattern github.com/GoogleCloudPlatform/workflows-demos/tree/master/retries-and-saga
15. Wait for HTTP/event callbacks instead of polling github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-translation github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-event
16. Parallelize when you can github.com/GoogleCloudPlatform/workflows-demos/tree/master/bigquery-parallel Orchestration usually involves steps run sequentially one after another. Try to parallelize those steps when you can. Example: running BigQuery jobs against Wikipedia dataset with Workflows: • Serial: 5 queries run sequentially each 20 seconds: Total 1 min • Parallel: 5 queries run in parallel: Total 20 seconds
17. Combine serverful workloads with serverless orchestration Sometimes you can’t use serverless due to some limitation (time, memory, CPU) Instead you use a Virtual Machine (VM) with the configuration you need Automate the VM lifecycle with an orchestrator to have a serverless experience github.com/GoogleCloudPlatform/workflows-demos/tree/master/long-running-container
18. Manage long running batch jobs with serverless orchestration github.com/GoogleCloudPlatform/workflows-demos/tree/master/screenshot-jobs github.com/GoogleCloudPlatform/batch-samples/tree/main/primegen
19. Use GitOps to manage orchestration lifecycle github.com/GoogleCloudPlatform/workflows-demos/tree/master/gitops
20. Plan for multi-environment deployments github.com/GoogleCloudPlatform/workflows-demos/tree/master/multi-env-deployment
21. Thank you! Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel Feedback? bit.ly/atamel