Memory API : Patterns, Performance et Cas d'Utilisation

La manière d'accéder à la mémoire off-heap d'une application n'a quasiment pas changé depuis 2002, quand les ByteBuffer ont été ajoutés au JDK 4. De nombreuses applications ont pourtant besoin de cette fonctionnalité, afin de stocker ou de traiter des grands volumes de données. Alors que les systèmes ont été étendus à 64 bits et que la mémoire se compte à présent en teraoctets, nous en sommes toujours à utiliser des ByteBuffers indexés sur 32 bits. Il était plus que temps qu'une nouvelle API arrive: c'est le projet Panama qui nous l'apportedans le JDK 22. Cette API permet d'accéder aux librairies natives d'une part, et à la mémoire off-heap d'autre part, objet de cette présentation.
Elle apporte de nouveaux concepts : les Arenas et les MemorySegments qui donnent accès à des zones de mémoire contiguës indexées sur 64 bits, ou encore les MemoryLayout, qui permettent d'accéder à cette mémoire à la façon des Struct du C.
Nous verrons comment l'API est organisée, les patterns qu'elle propose, et les performances que l'on peut en attendre. Nous verrons enfin comment accéder aux données au travers des VarHandle et comment utiliser Jextract pour simplifier l'écriture du code.
Les slides sont en anglais.

Transcript

1. Memory API: Patterns, Uses Cases, and Performance From the Panama Foreign Functions and Memory API José Paumard Java Developer Advocate Java Platform Group Rémi Forax Maître de conferences Université Gustave Eiffel
2. https://twitter.com/Nope! https://github.com/forax https://speakerdeck.com/forax OpenJDK, ASM, Tatoo, Pro, etc… One of the Father of invokedynamic (Java 7) Lambda (Java 8), Module (Java 9) Constant dynamic (Java 11) Record, text blocks, sealed types (Java 14 / 15) Valhalla (Java 25+)
3. https://dev.java @josepaumard.bsky.social
4. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 4 https://dev.java/
5. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 5 Tune in! Inside Java Newscast JEP Café Road To 21 series Inside.java Inside Java Podcast Sip of Java Cracking the Java coding interview
6. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 6 https://openjdk.org/ OpenJDK is the place where it all happens
7. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 7 https://jdk.java.net/ OpenJDK is the place where it all happens
8. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 8 Panama
9. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 9 First Preview in the JDK 14
10. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 10 Final Feature in the JDK 22
11. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 11 Final Feature in the JDK 22 https://openjdk.org/projects/panama/
12. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 12 Demo time!
13. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 13 Heal the rift between Java and C Fixing issues in the Java NIO API Namely, fix and update what you can do with ByteBuffer ByteBuffer where released in Java 4, in 2002 What is Panama About?
14. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 14 New assert keyword Exception chaining XML Parser Java NIO! (New Input / Output, JSR 51) What’s New in Java 4? (2002)
15. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 15 New assert keyword Exception chaining XML Parser Java NIO What’s New in Java 4?
16. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 16 Dynamic Web Site in 2002 IE 5.5 SP2? IE 6? ActiveXObject("Microsoft.XMLHTTP") Tomcat 3 Maybe 4?
17. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 17 The World Before Java 4 heap memory native memory write() read() array[i] array[i] Servlet doPost() doGet() ... COPY!
18. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 18 The World with NIO heap memory native memory write() read() buf.put() buf.get() ByteBuffer position limit capacity ... Servlet doPost() doGet() ...
19. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 19 Off-heap allocation File mapping Creating a ByteBuffer var buffer = ByteBuffer.allocateDirect(1_024); // int var buffer = FileChannel.map( READ_WRITE, position, size); // longs
20. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 20 Issues with ByteBuffers
21. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 21 Demo time!
22. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 22 Too high level for a Memory Access API position, capacity, reset are not needed 32 bits indexing only Allow for unaligned access, but may be very slow Non-deterministic deallocation! closing a mapped file does not close the ByteBuffer Issues with the ByteBuffer API
23. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 23 The GC - selects a region containing the ByteBuffer - then sees that the ByteBuffer is dead - then a Cleaner code (weakref) is pushed to a Cleaner queue Later, a cleaner thread dequeues the Cleaner code and calls free on the off-heap memory (or not…) How is Deallocation Working?
24. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 24 New API: the MemorySegment API - lower level than the ByteBuffer API - ByteBuffer are now built on top of MemorySegment Goals: - fix ByteBuffer issues - better interaction with C code Welcome to Panama
25. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 25 Introducing MemorySegment
26. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 26 A MemorySegment: - is safe (cannot be used once freed) - gives you control over the allocation / deallocation - brings close to C performance (and Unsafe) - offers direct access, indexed 64 bits access, structured access - opt-in unsafe access (for C interop, may crash later) - retrofit ByteBuffer on top MemorySegment
27. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 27 It is unsafe! - Close to C performance - No use after free protection (security) - Can peek/poke everywhere (may crash later) - No null check for on heap array access (may crash) Memory access methods are - deprecated for removal (JEP 471, del. JDK 23) - warnings since 2006 What about sun.misc.Unsafe?
28. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 28 Demo time!
29. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 29 Most CPU require your data to be aligned in memory These are properly aligned Alignement 0x00FFA000 0x00FFA004 0x00FFA008 0x00FFA00C 0x00FFA010 byte short int short byte
30. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 30 Most CPU require your data to be aligned in memory These are misaligned Alignement 0x00FFA000 0x00FFA004 0x00FFA008 0x00FFA00C 0x00FFA010 short int
31. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 31 Off-heap allocation, direct access The MemorySegment API var arena = Arena.global(); var segment = arena.allocation(1_024L); // long, off-heap segment.set(ValueLayout.JAVA_INT, 4L, 42); var value = segment.get(ValueLayout.JAVA_INT, 4L);
32. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 32 Heap allocation, indexed access The MemorySegment API var ints = new int[] {1, 2, 3, 4}; var segment = MemorySegment.ofArray(ints); // on-heap segment.setAtIndex(ValueLayout.JAVA_INT, 2L, 65); var cell = segment.getAtIndex(ValueLayout.JAVA_INT, 2L);
33. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 34 Copy from on-heap to off-heap The MemorySegment API: File Mapping var ints = new int[] {1, 2, 3, 4}; var arraySegment = MemorySegment.ofArray(ints); // on-heap var offHeapSegment = Arena.global().allocate(64L); // off-heap offHeapSegment.copyFrom(arraySegment);
34. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 35 Writing data to a mapped file: you need a ByteBuffer The MemorySegment API: File Mapping var ints = new int[] {1, 2, 3, 4}; var arraySegment = MemorySegment.ofArray(ints); // on-heap var offHeapSegment = Arena.global().allocate(64L); // off-heap offHeapSegment.copyFrom(arraySegment); var byteBuffer = offHeapSegment.asByteBuffer(); // this is a view! byteBuffer.limit( ints.length * (int) ValueLayout.JAVA_INT.byteSize()); try (var file = FileChannel.open(path, CREATE, WRITE)) { file.write(byteBuffer); }
35. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 36 Introducing Arena to Allocate / Deallocate
36. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 37 Demo time!
37. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 38 An Arena can create off-heap memory segments It initializes memory segment with zeroes It is AutoCloseable (more on this in a mn) It deallocates the memory segments it created on close() Introducing Arena
38. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 39 There are four of them: What is this Arena object? var global = Arena.global(); // singleton var confined = Arena.ofConfined(); var shared = Arena.ofShared(); var auto = Arena.ofAuto(); // supports legacy // ByteBuffer semantics
39. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 40 There are four of them: What is this Arena object? Bounded Lifetime Closed by the User Shared among threads Global No No Yes Auto Confined Shared
40. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 41 There are four of them: What is this Arena object? Bounded Lifetime Closed by the User Shared among threads Global No No Yes Auto Yes No (GC) Yes Confined Shared
41. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 42 There are four of them: What is this Arena object? Bounded Lifetime Closed by the User Shared among threads Global No No Yes Auto Yes No (GC) Yes Confined Yes Yes No Shared
42. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 43 There are four of them: What is this Arena object? Bounded Lifetime Closed by the User Shared among threads Global No No Yes Auto Yes No (GC) Yes Confined Yes Yes No Shared Yes Yes Yes
43. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 44 int[] vs memorySegment.get(JAVA_INT, ...) Benchmarks Array 0.728 ± 0.009 ns/op OfArray 1.358 ± 0.003 ns/op Confined 1.255 ± 0.002 ns/op Auto 1.254 ± 0.002 ns/op Shared 1.254 ± 0.013 ns/op Global 1.258 ± 0.026 ns/op Unsafe 0.627 ± 0.001 ns/op
44. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 45 Looping and summing 512 ints Benchmarks Array 128.338 ± 0.084 ns/op OfArray 131.927 ± 0.761 ns/op Confined 131.829 ± 0.077 ns/op Auto 131.832 ± 0.491 ns/op Shared 131.760 ± 0.068 ns/op Global 131.727 ± 0.137 ns/op Unsafe 128.083 ± 0.131 ns/op
45. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 46 Access time is independent of the type of arena For random direct access - Overhead is important (2x) - 3 checks 1. Is it the right thread? 2. Has the Arena been Closed? 3. Is access in bounds? Random Access Performance
46. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 47 Access time is independent of the type of arena For loop + indexed access - Fixed cost at the beginning of the loop - 3 Checks are hoisted out of the loop 1. Is it the right thread? Is done once 2. Has the Arena been Closed? Is done once 3. Is access in bounds? Is elided Loop Performance
47. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 48 Memory fragmentation Application integrity Jextract Memory Layout Structured memory access with offsets and VarHandle Lazy allocation using Stable Value (prev. 25) After the Break
48. Coffee Break!
49. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 50 About Arenas: - Different arena types with different semantics - Same access time What about allocation / deallocation? Welcome Back!
50. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 51 Allocation/Deallocation of an Arena + MemorySegment Benchmarks Array 2.522 ± 0.015 ns/op OfArray 6.494 ± 0.093 ns/op Confined 82.287 ± 1.530 ns/op Unsafe (malloc) 22.834 ± 0.097 ns/op Unsafe with init 72.338 ± 0.390 ns/op
51. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 52 With two arenas How Do Arenas Allocate Segments? var arenaA = Arena.ofConfined(); var a1 = arenaA.allocate(...); heap memory native memory Arena JVM a1
52. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 53 With two arenas How Do Arenas Allocate Segments? var arenaA = Arena.ofConfined(); var a1 = arenaA.allocate(...); var a2 = arenaA.allocate(...); heap memory native memory Arena JVM a1 a2
53. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 54 With two arenas How Do Arenas Allocate Segments? var arenaA = Arena.ofConfined(); var arenaB = Arena.ofConfined(); var b1 = arenaB.allocate(...); var a1 = arenaA.allocate(...); var b2 = arenaB.allocate(...); var a2 = arenaA.allocate(...); var b3 = arenaB.allocate(...); var b4 = arenaB.allocate(...); memory b2 a1 b3 a2 b4 b1
54. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 55 Then arenaA is closed And all its memory segments are deallocated How Do Arenas Allocate Segments? memory arenaA.close(); b2 b3 b4 b1 a1 a2
55. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 56 Then arenaC is created How Do Arenas Allocate Segments? memory var arenaC = Arena.ofConfined(); var c1 = arenaC.allocate(...); b2 b3 b4 c1 b1
56. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 57 Then arenaC is created How Do Arenas Allocate Segments? memory var arenaC = Arena.ofConfined(); var c1 = arenaC.allocate(...); var c2 = arenaC.allocate(...); c2 X X b2 b3 b4 b1 c1
57. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 58 Then arenaC is created And you end up with fragmentation! How Do Arenas Allocate Segments? memory var arenaC = Arena.ofConfined(); var c1 = arenaC.allocate(...); var c2 = arenaC.allocate(...); c2 b2 b3 b4 b1 X X c1
58. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 59 Holes in native memory It leads to two problems: - Allocation is slow, you need to find a large enough space for your memory segment - Not enough contiguous free memory may prevent the creation of a large memory segment Fragmentation
59. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 60 Default arenas create fragmentation But Arena is an interface! So you can implement your own allocation strategy https://docs.oracle.com/en/java/javase/24/docs/api/java.base/j ava/lang/foreign/Arena.html#custom-arenas Custom Arenas
60. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 61 Allocation/Deallocation of an Arena + MemorySegment Benchmarks Array 2.522 ± 0.015 ns/op OfArray 6.494 ± 0.093 ns/op Confined 82.287 ± 1.530 ns/op Auto 434.694 ± 247.868 ns/op Shared 6696.144 ± 37.833 ns/op Unsafe (malloc) 22.834 ± 0.097 ns/op Unsafe with init 72.338 ± 0.390 ns/op
61. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 62 ofAuto(): close() has the same semantics as ByteBuffer Slooooow! In the worst case scenario it calls System.gc() (even slooooooower!) Deallocation / close()
62. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 63 ofShared(): you want to avoid having a volatile access in get() (to know if the arena has been closed) close() performs a VM Handcheck with all other threads checks method on top of the stack is annotated as performing an access checks if the locals contains the closing arena ⇒ Linear with the number of platform threads Deallocation / close()
63. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 64 Confined: default choice, manual deallocation Shared = confined + multi thread access Not the malloc API, you should try to group allocations Global: permanent memory Auto: legacy, multi thread access, GC triggered deallocation Arena Uses Cases
64. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 65 Demo time!
65. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 66 Application Integrity
66. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 67 Memory Integrity is a Big Deal!
67. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 68 Memory segments are safe by default - even creating a memory segment from a long is safe (byteSize is 0) MemorySegment.reinterpret(newSize) - opt-in to unsafe, only for native memory - requires --enable-native-access on the command line - emits a warning in Java 23, will be an error in the future Unsafe MemorySegment
68. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 69 Draft JEP: Integrity by Default
69. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 70 JEP 261: Module System JEP 260: Encapsulate Most Internal APIs JEP 396: Strongly Encapsulate JDK Internals by Default JEP 403: Strongly Encapsulate JDK Internals JEP 451: Prepare to Disallow the Dynamic Loading of Agents JEP 471: Deprecate the Memory-Access Methods in sun.misc.Unsafe for Removal JEP 472: Prepare to Restrict the Use of JNI JEP 498: Warn upon Use of Memory-Access Methods in sun.misc.Unsafe Draft JEP: Integrity by Default
70. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 71 Demo time!
71. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 72 Jextract and MemoryLayout
72. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 73 Simplify C interoperability - Jextract takes a .h file and creates java classes from it - It creates one class for the .h with the function definitions - Then one per struct What is Jextract?
73. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 74 Uses LLVM internally To correctly parse C declarations And to extract platform/OS definitions (eg: what is the size of an int) It is an external tool, that needs to be downloaded separately https://jdk.java.net/jextract What is Jextract?
74. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 75 It is an interface that describe a piece of memory What is MemoryLayout? MemoryLayout SequencedLayout GroupLayout PaddingLayout ValueLayout StructLayout UnionLayout
75. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 76 Defining a struct Point A MemoryLayout can be Named var pointLayout = MemoryLayout.structLayout( ValueLayout.JAVA_INT, ValueLayout.JAVA_INT );
76. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 77 Size of the struct Point, offset of x and y in Point MemoryLayout Size and Offset var pointLayout = MemoryLayout.structLayout( ValueLayout.JAVA_INT, ValueLayout.JAVA_INT ); long pointLayoutSize = pointLayout.byteSize(); long xOffset = pointLayout.byteOffset(0); // by index
77. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 78 Size of the struct Point, offset of x and y in Point MemoryLayout Size and Offset var pointLayout = MemoryLayout.structLayout( ValueLayout.JAVA_INT.withName("x"), ValueLayout.JAVA_INT.withName("y") ).withName("point"); long pointLayoutSize = pointLayout.byteSize(); long xOffset = pointLayout.byteOffset(0); // by index long yOffset = pointLayout.byteOffset("y"); // by name
78. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 79 Memory layouts need padding Alignment and Padding struct { char kind; int payload; char extra; } 0 32 ki 64 16 48 payload padding static final MemoryLayout LAYOUT = MemoryLayout.structLayout( ValueLayout.JAVA_BYTE.withName("kind"), MemoryLayout.paddingLayout(3), ValueLayout.JAVA_INT.withName("payload"), ValueLayout.JAVA_BYTE.withName("extra"), MemoryLayout.paddingLayout(3) ); 80 96 ex padding
79. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 80 Demo time!
80. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 81 VarHandle
81. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 82 An object that gives access to fields with different semantics: - a get / set access (plain, opaque, volatile) - and concurrent access: compareAndSet, getAndAdd, … It hides the offset and size computations to access the elements of your memory layout What is a VarHandle?
82. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 83 Demo time!
83. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 84 1) The compiler doesn’t do any type checking: it trusts you! (and the different IDE are not there to hep you…) 2) It allows conversion at runtime It can be convenient, but can lead to autoboxing You can use withInvokeExactBehavior() VarHandle Caveats
84. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 85 Compute the sum of 512 point.x + point.y OfArray with offset 214.479 ± 1.712 ns/op OfArray with VarHandle 141.404 ± 0.081 ns/op Unsafe with offset 137.518 ± 0.476 ns/op Arena with offset 212.881 ± 3.436 ns/op Arena with VarHandle 141.190 ± 0.107 ns/op Benchmarks
85. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 86 Offset computation by the user is slow VarHandle offers an access pattern to the JVM, which gives you better performance Jextract does not create VarHandles, only offsets Using Offsets or VarHandle?
86. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 87 Stable Values
87. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 88 A stable value holds an eventually non-modifiable data Three guarantees: 1) Initialized when the value is first requested Not initialized at application startup nor at class initialization 2) Initialization code is run once 3) Once initialized, treated as (a real) constant by the JVM Stable Value
88. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 89
89. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 90 Current state of the high-level API (prev. in 25) Stable Value class StableValue { static <T> Supplier<T> supplier(Supplier<? extends T> supplier) { ... } static <T> List<T> list(int size, IntFunction<? extends T> mapper) { ... } static <K, V> Map<K, V> map(Set<K> keys, Function<? super K, ? extends V> mapper) { ... } }
90. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 91 Java 24 confinedStableMapLoop avgt 5 140.513 ± 0.132 ns/op confinedStableValueLoop avgt 5 140.973 ± 0.468 ns/op confinedVarHandleLoop avgt 5 140.862 ± 0.583 ns/op Java 25 (EA) confinedStableMapLoop avgt 5 22294.893 ± 99.458 ns/op confinedStableValueLoop avgt 5 140.634 ± 0.319 ns/op confinedVarHandleLoop avgt 5 140.533 ± 0.082 ns/op Stable Value – Performance
91. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 92 Stable Value https://cr.openjdk.org/~pminborg/stable- values2/api/java.base/java/lang/StableValue.html
92. 4/15/2025 Copyright © 2025, Oracle and/or its affiliates 93 Demo time!
93. Panama rocks!