MSA Resilient 2025

MSA 의 Resilient 를 구현하는 방법 with Resilience4j

1. sunghyouk.bae@gmail.com - ਷ృೠ Backend Developer, 2025-02-11. MSA Resilient Resilience4j
2. Agenda • Reactive Programming Manifesto • MSA Architecture • MSA ೙ࣻ ಁఢੋ Resilient patterns • Resilient ҳഅ୓ • Hystrix • Resilience4j • bluetape4k-resilience4j Generated by ChatGPT
3. Resilience
4. Reactive Programming Manifesto https://www.reactivemanifesto.org/ko
5. Reactive Programming Manifesto https://www.reactivemanifesto.org/ko
6. MSA Architecture Overview
7. Microservice Architecture Service Mesh https://istio.io/ https://docs.aws.amazon.com/ko_kr/app-mesh/latest/userguide/what-is-app-mesh.html https://www.consul.io/docs/intro
8. MSA - Resilent Service A Service B Service C Our Service Event Loop Asynchronous Non-Blocking Async Monitoring Fail? -> Retry? Fallback? Non-Blocking CompletableFuture ? Observable (RxJava) ? Cache
9. — Merriam-Webster Dictionary resilience: “An ability to recover from or adjust easily to misfortune or change.”
10. Resilient ೙ࣻ ӝמ • ੢গী ૒ݶೞ؊ۄب ਽׹ࢿਸ ਬ૑೧ঠ ೠ׮ • ੢গ ਃࣗী ؀਽೧ ࠂઁ, ࠉࣧ, Ѻܻ, ਤ੐ਸ ా೧ ੢গܳ ܻ࠙ೠ׮ • ؀಴ ӝמ • Circuit Breaker • Retry • Fallback • Limiters (Rate, Time)
11. Circuit Breaker Closed Open Half Open Call pass through count fail/success calls pass through
12. Circuit Breaker Closed Open Half Open trip breaker Call pass through count fail/success calls pass through
13. Circuit Breaker Closed Open Half Open trip breaker Call pass through count fail/success calls pass through
14. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success calls pass through
15. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success calls pass through
16. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success try reset calls pass through
17. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success try reset after timeout is reached calls pass through
18. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success try reset after timeout is reached calls pass through
19. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success trip breaker try reset after timeout is reached calls pass through
20. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success trip breaker try reset after timeout is reached calls pass through on failure
21. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success trip breaker try reset after timeout is reached calls pass through on failure
22. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success reset breakers trip breaker try reset after timeout is reached calls pass through on failure
23. Circuit Breaker Closed Open Half Open trip breaker If threshold reached Call pass through count fail/success reset breakers trip breaker try reset after timeout is reached calls pass through on success reset breaker on failure
24. Circuit Breaker • Close State • ੿࢚࢚కܳ աఋմ׮ • ੢গо ೲਊߧਤܳ ֈযࢲݶ, Open State۽ ੹ജ • Open State • ੢গ ࢚ട੉޲۽, ਃ୒ਸ ૊द Ѣࠗೠ׮ (੢গ Ѻܻ) • ౠ੿ दр, ഐ୹ പࣻ ੉റী ੢গ ࢚ട ੼Ѩਸ ਤ೧ Half Open State۽ ੹ജ • Half Open State • ੢গ ࢚ട੉ ೧ઁغ঻਺ਸ ੼Ѩೞӝ ਤೠ ઺р ࢚క (рࠁӝ ࢚ട) • ੢গ ࢚ട੉ ইפۄݶ Close State۽ ੹ജ (੿࢚), ই૒ ੢গ ࢚ട੉ۄݶ Open State۽ ੹ജ (੢গ)
25. Resilient ҳഅ୓ٜ
26. Net fl ix Hystrix Hystrix Flow chart Cache Reactive Async Sync Circuit Breaker Fallback Thread Pool
27. Net fl ix Hystrix Request caching & collapsing Request Caching Request Collapsing
28. Resilience4j • Hystrix ח ؊ ੉࢚ ӝמ ୶о হ׮ (maintenance mode) • Resilience4j ח ੄ઓࢿ੉ ୭ࣗചػ о߶਍ ۄ੉࠳۞ܻ • Bulkhead, Circuit breaker, Rate limiter, Retry, Time limiter, Cache ١ਸ ૑ਗ • Application ীࢲ ݆੉ ࢎਊೞח ۄ੉࠳۞ܻী ؀ೠ Adapter ઁҕ • Retro fi t2, Feign, Reactor, Micrometer … • Kotlin Coroutines ૑ਗ • resilience4j-kotlin, bluetape4k-resilience4j
29. Resilience4j Decorator patterns for CircuitBreaker
30. Resilience4j Decorator patterns for CircuitBreaker val breaker = breakerRegistry.circuitBreaker(“supplier”) // decorate simple supplier val supplier = breaker.checkedSupplier { "This can be any method which returns: `Hello" } val result = runCatching { supplier() }.map { "$it world`" }.recover { _ -> "Failed" } result.isSuccess.shouldBeTrue()
31. Resilience4j Decorator patterns for CircuitBreaker val circuitBreaker = CircuitBreaker.ofDefaults("test") val helloWorldService = CoHelloWorldService() // decorate suspend function val function = circuitBreaker.decorateSuspendFunction { helloWorldService.returnHelloWorld() } function() shouldBeEqualTo "Hello world" helloWorldService.invocationCount shouldBeEqualTo 1 val breaker = breakerRegistry.circuitBreaker(“supplier”) // decorate simple supplier val supplier = breaker.checkedSupplier { "This can be any method which returns: `Hello" } val result = runCatching { supplier() }.map { "$it world`" }.recover { _ -> "Failed" } result.isSuccess.shouldBeTrue()
32. Resilience4j Retry
33. Resilience4j Retry val service = mockk<Service>(relaxUnitFun = true) every { service.sayHello() } throws IOException(“Boom!") val supplier = retry.checkedSupplier(service::sayHello) val result = runCatching { supplier() }.recover { "Hello word from recovery execute" } val maxAttemps = retry.retryConfig.maxAttempts verify(exactly = maxAttemps) { service.sayHello() } confirmVerified(service) result.isSuccess.shouldBeTrue() result.getOrThrow() shouldBeEqualTo "Hello word from recovery execute"
34. Resilience4j Retry val service = mockk<Service>(relaxUnitFun = true) every { service.sayHello() } throws IOException(“Boom!") val supplier = retry.checkedSupplier(service::sayHello) val result = runCatching { supplier() }.recover { "Hello word from recovery execute" } val maxAttemps = retry.retryConfig.maxAttempts verify(exactly = maxAttemps) { service.sayHello() } confirmVerified(service) result.isSuccess.shouldBeTrue() result.getOrThrow() shouldBeEqualTo "Hello word from recovery execute" val retry = Retry.ofDefaults(“testName") val helloWorldService = CoHelloWorldService() val function = retry.decorateSuspendFunction { helloWorldService.returnHelloWorld() } function() shouldBeEqualTo "Hello world" helloWorldService.invocationCount shouldBeEqualTo 1
35. Resilience4j Support Kotlin Flow
36. Resilience4j Support Kotlin Flow flow { repeat(3) { emit(it) } } .circuitBreaker(circuitBreaker) .toList(results)
37. Resilience4j Support Kotlin Flow flow { repeat(3) { emit(it) } } .circuitBreaker(circuitBreaker) .toList(results) val helloWorldService = CoHelloWorldService() val retry = Retry.of("testName") { RetryConfig.custom<Any?>() .waitDuration(Duration.ofMillis(10)) .retryOnResult { helloWorldService.invocationCount < 2 } .build() } val results = flow { emit(helloWorldService.returnHelloWorld()) } .retry(retry) .toList() results.size shouldBeEqualTo 1 results[0] shouldBeEqualTo "Hello world"
38. Resilience4j Support Retro fi t2, Feign // Create a CircuitBreaker val circuitBreaker = CircuitBreaker.ofDefaults(”testName”); const val TIMEOUT = 300L; // ms val client = new OkHttpClient.Builder() .connectTimeout(TIMEOUT, TimeUnit.MILLISECONDS) .build(); val retrofit = new Retrofit.Builder() .addCallAdapterFactory(CircuitBreakerCallAdapter.of(circuitBreaker)) .baseUrl("http://localhost:8080/") .client(client) .build()
39. Resilience4j Support Spring Boot 2, 3 resilience4j.retry: configs: default: maxAttempts: 3 waitDuration: 100 retryExceptions: - org.springframework.web.client.HttpServerErrorException - java.util.concurrent.TimeoutException - java.io.IOException ignoreExceptions: - io.bluetape4k.workshop.resilience.exception.BusinessException instances: backendA: baseConfig: default backendB: baseConfig: default
40. Resilience4j Support Spring Boot 2, 3 @TimeLimiter(name = BACKEND_A) @CircuitBreaker(name = BACKEND_A, fallbackMethod = "fluxFallback") override fun fluxTimeout(): Flux<String> { return Flux.just("Hello World as Flux from backend A") .delayElements(Duration.ofSeconds(3)) // time limiter о 2s ۽ ࢸ੿غয ੓ਵ޲۽, timeout ੉ ߊࢤ೤פ׮. } @TimeLimiter(name = BACKEND_A) @Bulkhead(name = BACKEND_A) @CircuitBreaker(name = BACKEND_A, fallbackMethod = "monoFallback") override fun monoTimeout(): Mono<String> { log.debug { "Mono Timeout..." } return Mono.just("Hello World as Mono from backend A") .delayElement(Duration.ofSeconds(3)) // time limiter о 2s ۽ ࢸ੿غয ੓ਵ޲۽, timeout ੉ ߊࢤ೤פ׮. }
41. Resilience4j Support Kotlin Coroutines private val circuitBreaker = circuitBreakerRegistry.circuitBreaker(BACKEND_B) private val retry = retryRegistry.retry(BACKEND_B) @GetMapping("suspendFallback") suspend fun suspendFallback(): String = executeWithFallback( block = { serviceB.suspendFailure() }, fallback = { e -> "Fallback: ${e?.message}" } ) @GetMapping("flowFailure") fun flowFailure(): Flow<String> = executeFlow { serviceB.flowFailure() } private suspend fun <T> executeWithFallback(block: suspend () -> T, fallback: (Throwable?) -> T): T { return CoDecorators.ofSupplier(block) .withCircuitBreaker(circuitBreaker) .withBulkhead(bulkhead) .withRetry(retry) .withTimeLimiter(timeLimiter) .withFallback { e: Throwable? -> fallback(e) } // fallback ӝמ ୶о .get() } private fun <T> executeFlow(block: () -> Flow<T>): Flow<T> { return block() .bulkhead(bulkhead) .circuitBreaker(circuitBreaker) .retry(retry) }
42. bluetape4k resilience4j bluetape4k-resilience4j module • Resilience4j о ҕध੸ਵ۽ ઁҕೞ૑ ঋח Coroutines ਊ `Decorators` ઁҕ • CoDecorators • CompletableFutureܳ ߈ജೞೞח ೣࣻী ؀ೠ Decorator ૑ਗ • Var ࠁ׮ ಞೠ Kotlin ߑध੄ recover ೣࣻ ઁҕ (fallback) • Suspend ೣࣻী ؀ೠ cache ૑ਗ • CoroutineCache
43. bluetape4k-resilience4j coEvery { service.supply() } coAnswers { expected } val decorated = CoDecorators .ofSupplier { service.supply() } .withRetry(retry) .withCircuitBreaker(circuitBreaker) .decoreate() val result = runCatching { decorated() } result.isSuccess.shouldBeTrue() result.getOrNull() shouldBeEqualTo expected coVerify(exactly = 1) { service.supply() } confirmVerified(service) coEvery { service.consume(input) } returns Unit val decorated = CoDecorators .ofRunnable { service.consume(input) } .withRetry(retry) .withCircuitBreaker(circuitBreaker) .decoreate() val result = runCatching { decorated() } result.isSuccess.shouldBeTrue() coVerify(exactly = 1) { service.consume(input) } confirmVerified(service)
44. bluetape4k-resilience4j Support cache for suspend function val _called = atomic(0) val called by _called val function: suspend (String) -> String = { name: String -> _called.incrementAndGet() log.debug { "Cache function invoked. name=$name, called=$called" } greeting(name) } val cachedFunc = cache.decorateSuspendedFunction(function) val key1 = randomKey("key") val key2 = randomKey("key") cachedFunc(key1) shouldBeEqualTo "Hi $key1!" called shouldBeEqualTo 1 cachedFunc(key2) shouldBeEqualTo "Hi $key2!" called shouldBeEqualTo 2 cachedFunc(key1) shouldBeEqualTo "Hi $key1!" called shouldBeEqualTo 2 cachedFunc(key2) shouldBeEqualTo "Hi $key2!" called shouldBeEqualTo 2
45. хࢎ೤פ׮