Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Project Loom: Broadening Concurrency

Mark Paluch
September 23, 2022

Project Loom: Broadening Concurrency

Project Loom has landed in Java 19 as a preview feature. It promises to simplify concurrency the way garbage collection simplifies memory management and improves applications' scalability.

It's about time to closely examine the value proposition and how it is implemented to understand what Loom can provide for your application. We will dive into technical details to understand concurrency, look at examples of how they behave, and how we can debug virtual thread code. You will learn what to watch out for and how project Loom can help you to write simple, concurrent code to get the most out of your runtime environment.

Mark Paluch

September 23, 2022
Tweet

More Decks by Mark Paluch

Other Decks in Programming

Transcript

  1. Mark Paluch Spring Data Project Lead @ VMware R2DBC Spec

    Lead Lettuce Redis Driver Maintainer Paramedic & SAR Dogs
  2. What is Project Loom • Initiative to introduce virtual threads

    to the Java Platform. Virtual threads are lightweight threads that dramatically reduce the effort of writing, maintaining, and observing high-throughput concurrent applications.
  3. What is Project Loom • Initiative to introduce Introduction of

    virtual threads and concurrency API to the Java Platform. Virtual threads are lightweight threads that eliminate I/O wait. dramatically reduce the effort of writing, maintaining, and observing high-throughput concurrent applications.
  4. What Project Loom actually is • Introduction of virtual threads

    and concurrency API to the Java Platform. Virtual threads are lightweight threads that eliminate I/O wait.
  5. Dramatically reduce the effort of writing, maintaining, and observing concurrent

    applications • True if you run on a Virtual Thread • Future.get(), Thread.sleep(), Lock.lock() are good friends again • No need for future chaining, using complex reactive or asynchronous API • Not so, if you accidentally run on a Kernel Thread
  6. Carrier Threads • Separate Fork/Join Pool of Platform Threads •

    Default size: Runtime.availableProcessors() • Configurable through system properties (jdk.virtualThreadScheduler.parallelism, maxPoolSize, minRunnable)
  7. Virtual Threads API • Extended Thread API Thread.ofVirtual().start(() -> {…});

    Executors.newVirtualThreadPerTaskExecutor() Thread.currentThread().isVirtual() • Implemented almost entirely in Java
  8. try(var e = Executors.newVirtualThreadPerTaskExecutor()){ for (int j = 0; j

    < 100; j++) { e.execute(() -> Thread.sleep(100)); } } // implicit wait
  9. Parking & Unparking • On each blocking operation • If

    blocked, capture stack and store it • Carrier thread free to perform other work (unpark next virtual thread)
  10. Blocking in synchronized code • Leads to Carrier Thread Pinning

    • Carrier Thread bound to virtual thread until unblock • synchronized is a JVM behavior, cannot be controlled from Java code • I/O, Locks, Future.get() in synchronized leads to Thread Pinning
  11. Thread Type Selection • I/O-heavy, shared-resources with Java Locks, Concurrency

    Utils • Virtual Threads • Computation-heavy, synchronized workloads • Platform Threads
  12. Observability, Tooling, Trouble Shooting • Java Flight Recorder Events •

    Virtual Thread start/end, pinned, submission failed • Debugging: IntelliJ announced tooling support for later this year • Pinned thread monitoring • -Djdk.tracePinnedThreads=full|short
  13. Usage of Virtual Thread API • In every piece that

    could block or wait in Java • Thread.sleep • Locks, ConcurrentHashMap • I/O (Socket) • Rewrite of several components as pre-requisite for Loom to avoid thread pinning
  14. Usage Pattern • Kernel Threads („Platform Threads“) • Expensive •

    Thread Pooling • Virtual Threads • Cheap • New Virtual Thread per Runnable
  15. JMH Benchmarks Benchmark Mode Cnt Score Error Units VirtualThreadsBenchmark.createVirtualThreads thrpt

    5 14488555,637 ± 1127513,241 ops/s VirtualThreadsBenchmark.runAndAwait thrpt 5 455494,235 ± 22033,286 ops/s PlatformThreadsBenchmark.createPlatformThreads thrpt 5 19094,736 ± 677,739 ops/s PlatformThreadsBenchmark.runAndAwait thrpt 5 15807,802 ± 7337,906 ops/s
  16. JMH Benchmarks Benchmark Mode Cnt Score Error Units VirtualThreadsBenchmark.createVirtualThreads thrpt

    5 14488555,637 ± 1127513,241 ops/s VirtualThreadsBenchmark.runAndAwait thrpt 5 455494,235 ± 22033,286 ops/s PlatformThreadsBenchmark.createPlatformThreads thrpt 5 19094,736 ± 677,739 ops/s PlatformThreadsBenchmark.runAndAwait thrpt 5 15807,802 ± 7337,906 ops/s ThreadPoolExecutorBenchmark.runAndAwait thrpt 5 461335,862 ± 54427,010 ops/s
  17. What does that mean? • If the CPU availability is

    the limit: You cannot increase concurrency • If the Runnable wait is the limit: You can increase concurrency
  18. Scalability Boundaries • I/O in synchronized Code • Connection Pool

    Sizes • Databases • HTTP/JMS/… Connection Pools • File handles • number of open files
  19. Pitfalls • System.out.println(…) • Attempts to obtain a lock on

    the I/O • Measurement and benchmark correctness („What do you measure?“) • Visible and invisible Thread Pinning
  20. Client/Server Workloads • Workload to be served by virtual thread

    (carrier threads must be available) • System must have sufficient resources to serve the request (e.g. connection pools). Also a constraint for platform thread usage. • Request to server via I/O • During wait, virtual thread is parked • Client must not pin kernel thread (see carrier thread availability)
  21. Computation Workloads • Incoming request/event to be served by virtual

    thread (carrier threads must be available) • System performs a computation based on in-memory resources • Response is returned
  22. Call to Action • Investigate: What is limiting your application

    scalability? • Investigate: What workloads do you run? • Experiment: Run your application on Virtual Threads and benchmark it! • Observe: What libraries lead to thread pinning?