Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Java Concurrency from the Trenches - 2025

Java Concurrency from the Trenches - 2025

What happens when your batch job goes from “it works on my laptop” to triggering out-of-memory errors and DDoS-ing your own dependencies? In this talk, we’ll walk through the real-world evolution of a high-scale Spring Boot + gRPC workload — from a single-threaded prototype to a concurrency pipeline held together with virtual threads, semaphores, and hard lessons.

You’ll see what happens when parallelStream() backfires, how misused thread pools create bottlenecks, and how rate limiting inside your async tasks can stall your fanout. Through practical examples and code walk-throughs, we’ll explore the trade-offs between different concurrency tools — and why “just making it async” isn’t enough.

Key takeaways:
- Why parallelStream() isn’t always your friend (especially with nested structures or flatMap)
- How to safely mix CompletableFuture, thread pools, and blocking IO
- When to use RateLimiter, Semaphore, or both — and how to protect your dependencies and yourself
- What virtual threads actually solve (and what they don’t)

If you’ve ever added concurrency and made things worse, this talk is for you.

Avatar for Hugo Marques

Hugo Marques

July 10, 2025
Tweet

More Decks by Hugo Marques

Other Decks in Programming

Transcript

  1. About Me • 20+ years as a D&D Dungeon Master

    • Software Engineer with 10+ years of experience • Amazon/AWS, Twitter, Netflix • Focused on Product Development
  2. Who this talk is for ✅ Developers writing concurrent code

    in real applications ✅ Engineers facing scaling issues — especially with I/O ✅ People curious about concurrency and its challenges
  3. ❌ Not a benchmarking or performance deep dive ❌ Not

    about JVM or low-level app tuning ❌ Not a full reference of the Java concurrency API What this talk isn’t about
  4. Waiting on Networks Files DB gRPC CPU-bound vs. IO-bound Math

    Compression Tight Loops • • • • ◦ ◦ ◦
  5. | Thread | Old-school, rarely used now | | ExecutorService

    | Thread Pool management | | CompletableFuture | Non-blocking composition | | parallelStream | Easy concurrency for CPU work | | Virtual Threads | New, flexible, for IO-heavy flows | Java Concurrency Toolbox Tool When/Why |
  6. Orders Processing at Scale | Scheduler | 1 run |

    Every 30 min | | Regions | 10 | ~10 | | Orders | 10k per region | 100k total | | Products | ~5k per order | ~500M total | | gRPC Calls | 1 per 100 products | ~5M calls | Layer Units Notes | | Total: ~ 270k RPS
  7. ✅ Easy to understand ✅ Memory-safe ✅ Safe for our

    dependencies ❌ Slow ❌ Underutilized our resources ✅ It works!
  8. 🛠 Default parallelism: Runtime.getRuntime().availableProcessors() == # of processors - 1

    🧩 To customize it, manually use ForkJoinPool: Intro to Parallel Stream
  9. Solution #4: thenApply comparison | Where | Same thread as

    previous stage | Default(ForkJoinPool) | Provided executor | | Performance | Fastest (no context switch) | Slower (context switch)| Slower (context switch) | | Flexibility | Low | Medium | High | | Simplicity | High | Medium | Low | | Context | Preserved | Lost | Lost | thenApply thenApplyAsync thenApplyAsync with executor | | | However…
  10. Solution #5: Semaphore vs Rate Limiter | Controls | Concurrent

    calls | Call frequency | | Unit of measure | Max in-flight tasks | Max starts per second | | Protects | Your CPU, memory, threads | Your dependency | | When to use | Tasks run too long / build up | Too many calls too quickly | | Risk if missing | App overloads itself | You flood external services | Sometimes you need both… Aspect Semaphore (Bulkhead) RateLimiter | |
  11. Solution #7: JDK 24 Update to JDK24, or wait for

    next LTS version: JDK25 in September 2025
  12. Solution #9: Virtual Threads solution comparison | Controls | Coarse-control

    | Fine-grained | | Levers | At Order and GRPC level | At Order level only | | Semaphore size | Order semaphore = 8, grpc semaphore = 50 | Order semaphore = 50 | | Code complexity | Better than the IO version | Better | Aspect Virtual Threads at Order & Grpc level Virtual Threads at Order level | |
  13. Key Takeaways ✅ Experiment with and measure your solutions ✅

    Protect your resources and work backward from there ✅ Concurrency is hard, so keeping it simple is better