Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JVMs.Compare - Benchmark Performance - OCO19

Chandra Guntur
September 19, 2019

JVMs.Compare - Benchmark Performance - OCO19

Comparing benchmark performance of different JDK/JVMs

With the multiple options now available in the Java world, developers have a vast range of combinations when they choose the right JDK and JVM for their needs. This session will detail the benchmark performance of a java library with several combinations of JDK and JVMs.

GraalVM and OpenJ9 have taken the Java world by storm, given their promise of performance enhancements. With the multiple options now available in the Java world, developers have a vast range of combinations when they choose the right JDK and JVM for their needs. Most developers tend to run a limited set of relevant metrics to identify the “right mix,” and many others simply try to tweak flags in the current bundled implementation they use. This session, using both a popular Java collections library, Eclipse Collections, and a few JMH benchmarking options, presents a comparison of performance metrics on the collection API, using GraalVM, Oracle JDK 8, AdoptOpenJDK 8 with HotspotVM and OpenJ9.

Chandra Guntur

September 19, 2019
Tweet

More Decks by Chandra Guntur

Other Decks in Technology

Transcript

  1. @CGuntur @TheDonRaab !2 About Us 2 • Java Champions •

    JCP Executive Committee Reps. for BNY Mellon • Ardent bloggers and tweeters • Managing Director at BNY Mellon • 18+ years of experience in 
 Financial Tech. • Programming in Java since 1997 • Member of JSR 335 Expert Group • Creator of Eclipse Collection 
 Java Library • Director at BNY Mellon • Working in Financial Tech. 
 since 2003 • Programming in Java since 1998 • JUG Leader @ NYJavaSIG • Creator of Java-Katas Github 
 repository
  2. @CGuntur @TheDonRaab !3 Session Agenda Compare benchmarks for operations using

    
 Eclipse Collections and JDK’s Java Collection Framework: •on a primitive IntList & List<Integer> •on a List<Person> using several JDK/JVM combinations
  3. @CGuntur @TheDonRaab !4 Compared JDK/JVMs JDK/JVM combinations tested: 1. Oracle

    JDK 8 (v1.8.0_221) 2. GraalVM Enterprise Edition (v19.2.0) 3. GraalVM Community Edition (v19.2.0) 4. AdoptOpenJDK 8 w/Hotspot (v1.8.0_221) 5. AdoptOpenJDK 8 w/OpenJ9 (v1.8.0_221) 6. OpenJDK 11 (v11.0.2) 7. OpenJDK 11 embedded Graal JVMCI (v11.0.2) 8. GraalVM Enterprise Edition C2 Compiler (v19.2.0)
  4. @CGuntur @TheDonRaab !5 Benchmarks on IntList operations Three operations were

    benchmarked on primitive collections: •Filter: Filter in all even numbers (5 benchmarks)
  5. @CGuntur @TheDonRaab !5 Benchmarks on IntList operations Three operations were

    benchmarked on primitive collections: •Filter: Filter in all even numbers (5 benchmarks) •Sum: Sum of all integers in the collection (5 benchmarks)
  6. @CGuntur @TheDonRaab !5 Benchmarks on IntList operations Three operations were

    benchmarked on primitive collections: •Filter: Filter in all even numbers (5 benchmarks) •Sum: Sum of all integers in the collection (5 benchmarks) •Transform: Multiply each integer by 2 (5 benchmarks)
  7. @CGuntur @TheDonRaab !6 Benchmarks on Object operations Four operations were

    benchmarked on non-primitive collections: • Filter: Filter all Persons with height between 80 and 150 inches (7 benchmarks)
  8. @CGuntur @TheDonRaab !6 Benchmarks on Object operations Four operations were

    benchmarked on non-primitive collections: • Filter: Filter all Persons with height between 80 and 150 inches (7 benchmarks) • FilterAndGroup: Filter all Persons less than 150 inches tall, then group by age (7 benchmarks)
  9. @CGuntur @TheDonRaab !6 Benchmarks on Object operations Four operations were

    benchmarked on non-primitive collections: • Filter: Filter all Persons with height between 80 and 150 inches (7 benchmarks) • FilterAndGroup: Filter all Persons less than 150 inches tall, then group by age (7 benchmarks) • IntSummaryStatistics: Summary statistics by age (integer) of all Person instances (5 benchmarks)
  10. @CGuntur @TheDonRaab !6 Benchmarks on Object operations Four operations were

    benchmarked on non-primitive collections: • Filter: Filter all Persons with height between 80 and 150 inches (7 benchmarks) • FilterAndGroup: Filter all Persons less than 150 inches tall, then group by age (7 benchmarks) • IntSummaryStatistics: Summary statistics by age (integer) of all Person instances (5 benchmarks) • CombinedSummaryStatistics: Summary statistics on height (double), weight (double) and age (integer) (5 benchmarks)
  11. @CGuntur @TheDonRaab !7 JDK Flags - Part 1 In order

    to run the embedded Graal compiler in OpenJDK 11: • +XX:+UnlockExperimentalVMOptions • +XX:+EnableJVMCI
 • +XX:+UseJVMCICompiler
  12. @CGuntur @TheDonRaab !8 JDK Flags - Part 2 In order

    to run the C2 compiler in GraalVM EE v19.2.0: •+XX:+UnlockExperimentalVMOptions •+XX:-UseJVMCICompiler Notice the minus before the UseJVMCICompiler
  13. @CGuntur @TheDonRaab !9 Hardware - The Beast Model Name: Mac

    Pro Model Identifier: MacPro6,1 Processor Name: 12-Core Intel Xeon E5 Processor Speed: 2.7 GHz Number of Processors: 1 Total Number of Cores: 12 L2 Cache (per Core): 256 KB L3 Cache: 30 MB Memory: 64 GB
  14. @CGuntur @TheDonRaab !10 Model Name: MacBook Pro Model Identifier: MacBookPro11,1

    Processor Name: Intel Core i7 Processor Speed: 2.8 GHz Number of Processors: 1 Total Number of Cores: 2 L2 Cache (per Core): 256 KB L3 Cache: 4 MB Memory: 16 GB Hardware - The Little Monster
  15. @CGuntur @TheDonRaab !11 Benchmark Math Some relevant numbers in terms

    of what was run: 7 Benchmark classes 8 JDK/JVM combinations
  16. @CGuntur @TheDonRaab !11 Benchmark Math Some relevant numbers in terms

    of what was run: 7 Benchmark classes 8 JDK/JVM combinations 2 runs
  17. @CGuntur @TheDonRaab !11 Benchmark Math Some relevant numbers in terms

    of what was run: 7 Benchmark classes 8 JDK/JVM combinations 2 runs Mode.Throughput on collections, seeded with size = 1_000_000
  18. @CGuntur @TheDonRaab !11 Benchmark Math Some relevant numbers in terms

    of what was run: 7 Benchmark classes 8 JDK/JVM combinations 2 runs Mode.Throughput on collections, seeded with size = 1_000_000 Mode.Throughput on collections, seeded with size = 1000
  19. @CGuntur @TheDonRaab !11 Benchmark Math Some relevant numbers in terms

    of what was run: 7 Benchmark classes 8 JDK/JVM combinations 2 runs Mode.Throughput on collections, seeded with size = 1_000_000 Mode.Throughput on collections, seeded with size = 1000 39 benchmarks in the 7 classes = 2184 benchmarks.
  20. @CGuntur @TheDonRaab !11 Benchmark Math Some relevant numbers in terms

    of what was run: 7 Benchmark classes 8 JDK/JVM combinations 2 runs Mode.Throughput on collections, seeded with size = 1_000_000 Mode.Throughput on collections, seeded with size = 1000 39 benchmarks in the 7 classes = 2184 benchmarks. Executed twice = 4368 total benchmarks.
  21. @CGuntur @TheDonRaab !11 Benchmark Math Some relevant numbers in terms

    of what was run: 7 Benchmark classes 8 JDK/JVM combinations 2 runs Mode.Throughput on collections, seeded with size = 1_000_000 Mode.Throughput on collections, seeded with size = 1000 39 benchmarks in the 7 classes = 2184 benchmarks. Executed twice = 4368 total benchmarks. Throughput benchmarks - higher is better.
  22. @CGuntur @TheDonRaab !11 Benchmark Math Some relevant numbers in terms

    of what was run: 7 Benchmark classes 8 JDK/JVM combinations 2 runs Mode.Throughput on collections, seeded with size = 1_000_000 Mode.Throughput on collections, seeded with size = 1000 39 benchmarks in the 7 classes = 2184 benchmarks. Executed twice = 4368 total benchmarks. Throughput benchmarks - higher is better.
  23. @CGuntur @TheDonRaab !14 Throughput JVM/JDK 01 02 03 04 05

    06 07 08 X-axis = JDK/JVMs Y-axis = Throughput
  24. @CGuntur @TheDonRaab !14 Throughput JVM/JDK 01 02 03 04 05

    06 07 08 X-axis = JDK/JVMs Y-axis = Throughput Each line = @Benchmark
  25. @CGuntur @TheDonRaab !15 Collection Size 1M Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  26. @CGuntur @TheDonRaab !16 Collection Size 1K Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  27. @CGuntur @TheDonRaab !17 Collection Size 1M Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  28. @CGuntur @TheDonRaab !18 Collection Size 1K Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  29. @CGuntur @TheDonRaab !19 Collection Size 1M Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  30. @CGuntur @TheDonRaab !20 Collection Size 1K Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  31. @CGuntur @TheDonRaab !21 Collection Size 1M Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  32. @CGuntur @TheDonRaab !22 Collection Size 1K Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  33. @CGuntur @TheDonRaab !23 Collection Size 1M Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  34. @CGuntur @TheDonRaab !24 Collection Size 1K Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  35. @CGuntur @TheDonRaab !25 Collection Size 1M Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  36. @CGuntur @TheDonRaab !26 Collection Size 1K Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  37. @CGuntur @TheDonRaab !27 Collection Size 1M Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot
  38. @CGuntur @TheDonRaab !28 Collection Size 1K Throughput JVM/JDK 01 02

    03 04 05 06 07 08 01 = Oracle JDK 8 02 = GraalEE 19.2.0 03 = Graal Community 19.2.0 04 = AdoptOpenJDK8 Hotspot 05 = AdoptOpenJDK8 OpenJ9 06 = OpenJDK 11 Hotspot 07 = OpenJDK 11 Graal JVMCI 08 = Graal Embedded Hotspot