Java Concurrency from the Trenches - 2025

What happens when your batch job goes from “it works on my laptop” to triggering out-of-memory errors and DDoS-ing your own dependencies? In this talk, we’ll walk through the real-world evolution of a high-scale Spring Boot + gRPC workload — from a single-threaded prototype to a concurrency pipeline held together with virtual threads, semaphores, and hard lessons.

You’ll see what happens when parallelStream() backfires, how misused thread pools create bottlenecks, and how rate limiting inside your async tasks can stall your fanout. Through practical examples and code walk-throughs, we’ll explore the trade-offs between different concurrency tools — and why “just making it async” isn’t enough.

Key takeaways:
- Why parallelStream() isn’t always your friend (especially with nested structures or flatMap)
- How to safely mix CompletableFuture, thread pools, and blocking IO
- When to use RateLimiter, Semaphore, or both — and how to protect your dependencies and yourself
- What virtual threads actually solve (and what they don’t)

If you’ve ever added concurrency and made things worse, this talk is for you.

Transcript

1. Java Concurrency from the trenches Lessons learned in the wild by Hugo Marques
2. About Me • 20+ years as a D&D Dungeon Master • Software Engineer with 10+ years of experience • Amazon/AWS, Twitter, Netflix • Focused on Product Development
3. Who this talk is for ✅ Developers writing concurrent code in real applications ✅ Engineers facing scaling issues — especially with I/O ✅ People curious about concurrency and its challenges
4. ❌ Not a benchmarking or performance deep dive ❌ Not about JVM or low-level app tuning ❌ Not a full reference of the Java concurrency API What this talk isn’t about
5. Foundations
6. Sequential vs. Parallel vs. Concurrent
7. Synchronous vs. Asynchronous
8. Waiting on Networks Files DB gRPC CPU-bound vs. IO-bound Math Compression Tight Loops • • • • ◦ ◦ ◦
9. | Thread | Old-school, rarely used now | | ExecutorService | Thread Pool management | | CompletableFuture | Non-blocking composition | | parallelStream | Easy concurrency for CPU work | | Virtual Threads | New, flexible, for IO-heavy flows | Java Concurrency Toolbox Tool When/Why |
10. Our Problem
11. Orders Processing at Scale | Scheduler | 1 run | Every 30 min | | Regions | 10 | ~10 | | Orders | 10k per region | 100k total | | Products | ~5k per order | ~500M total | | gRPC Calls | 1 per 100 products | ~5M calls | Layer Units Notes | | Total: ~ 270k RPS
12. High-Level Architecture
13. First solution: Sequential
15. ✅ Easy to understand ✅ Memory-safe ✅ Safe for our dependencies ❌ Slow ❌ Underutilized our resources ✅ It works!
16. CPU bound solution: Parallel Stream
17. 🛠 Introduced with Java 8, allows streams Parallel execution: Intro to Parallel Stream
18. 🛠 Default parallelism: Runtime.getRuntime().availableProcessors() == # of processors - 1 🧩 To customize it, manually use ForkJoinPool: Intro to Parallel Stream
19. ⚠ Use ForkJoinPool.commonPool by default Intro to Parallel Stream ⚠ Shared globally — even with unrelated code
22. Problem #1: Nested Parallel Stream
23. Problem #1: Nested Parallel Stream
24. Solution #1: Parallelize where it matters most
25. Problem #2: A Flatmap Surprise The code above doesn’t run in parallel!
26. Problem #2: Why?
27. Solution #2: Reverse the order Parallelize where it matters most
28. Problem #3: Parallel Stream & Thread Context
29. Problem #3: Parallel Stream & Thread Context
30. IO bound solution: Executors & Completable Future
31. Executors & Completable Future The Plain Old Java
32. Executors & Completable Future The Spring Way (Option 1)
33. Executors & Completable Future The Spring Way (Option 2)
34. Executors & Completable Future The Spring Way (Option 3)
37. Actually…
39. What happened?
40. Problem #4: Our first OOM
42. Problem #4: Our first OOM A tale of shared thread pools
43. Problem #4: Our first OOM A tale of shared thread pools
44. Solution #4: Separate the pools
45. Solution #4: Separate the Pools with thenApply
46. Solution #4: Separate the Pools with thenApplyAsync
47. Solution #4: Separate the Pools thenApplyAsync + Executor
48. Solution #4: thenApply comparison | Where | Same thread as previous stage | Default(ForkJoinPool) | Provided executor | | Performance | Fastest (no context switch) | Slower (context switch)| Slower (context switch) | | Flexibility | Low | Medium | High | | Simplicity | High | Medium | Low | | Context | Preserved | Lost | Lost | thenApply thenApplyAsync thenApplyAsync with executor | | | However…
49. Problem #5: DDOS my dependencies MY DOWNSTREAM DEPENDENCY
50. Solution #5: Self-throttling — Semaphores & Rate Limiter Semaphores Rate Limiter
51. Solution #5: Semaphore vs Rate Limiter | Controls | Concurrent calls | Call frequency | | Unit of measure | Max in-flight tasks | Max starts per second | | Protects | Your CPU, memory, threads | Your dependency | | When to use | Tasks run too long / build up | Too many calls too quickly | | Risk if missing | App overloads itself | You flood external services | Sometimes you need both… Aspect Semaphore (Bulkhead) RateLimiter | |
52. Problem #6: OOM Strikes Back
53. Problem #6: OOM Strikes Back
54. Solution #6: Add a Semaphore to control backpressure
55. Let's Review Our Solution! Solution #6: Add a Semaphore to control backpressure
57. Can we make it simpler?
58. A non-blocking IO solution
64. A bounded-executor solution
68. Virtual Threads
69. The Main Idea
70. Enabling Virtual Threads with Spring
71. However…
72. Problem #7: The pinning problem
73. Problem #7: The pinning problem
74. Solution #7: JDK 24 Update to JDK24, or wait for next LTS version: JDK25 in September 2025
75. Problem #8: Yet another OOM
76. Problem #8: Yet another OOM
78. Solution #8: Back to the semaphore
79. Solution #8: Back to the semaphore
80. Results
81. Bounded-Executor vs Virtual Threads Job duration time with 3 nodes
82. Bounded-Executor vs Virtual Threads 16k 34k
83. Can we make it simpler?
84. Checking our metrics… 16k 70
85. Solution #9: A simpler virtual threads solution
86. Solution #9: Virtual Threads solution comparison | Controls | Coarse-control | Fine-grained | | Levers | At Order and GRPC level | At Order level only | | Semaphore size | Order semaphore = 8, grpc semaphore = 50 | Order semaphore = 50 | | Code complexity | Better than the IO version | Better | Aspect Virtual Threads at Order & Grpc level Virtual Threads at Order level | |
87. Solution #9: Virtual Threads solution comparison
88. Solution #9: Virtual Threads solution comparison 13k 49
90. Key Takeaways
91. Key Takeaways ✅ Experiment with and measure your solutions ✅ Protect your resources and work backward from there ✅ Concurrency is hard, so keeping it simple is better
92. Thank you! bsky.app/profile/hugomarques.dev References dev.to/hugaomarques www.linkedin.com/in/hugodesmarques/