Java. an evangelist and heavy user of Spring and Spring Boot. an organizer/presenter at NY Java Special Interest Group ( ). a co-chair of NYJavaSIG Hands On Workshop (NYJavaSIG HOW). committed to docendo discimus (by teaching, you learn). a Saganist ( the g not to be confused with a t ). Twitter: LinkedIn: My blog: LinkedIn and my blog QR codes generated at QRStuff: . javasig.com http://www.twitter.com/cguntur http://www.linkedin.com/in/chandraguntur http://www.cguntur.me http://www.qrstuff.com 2
is the process of: looking up managed memory identify all objects that are in use (live objects) mark non-live objects as garbage (unused or no-reference are other common terms) [occasionally] reclaim memory by deleting garbage [occasionally] compact memory defragmenting live objects (create more contiguous space). An unused object is one that is no longer referenced by any part of the program. History: The first garbage collection process: Lisp, in 1959 by John MacCarthy (author of Lisp and major contributor to ALGOL fame). 4 . 1
several classifiers. Few of those are tabulated below. Below numbering is only for readability. It does not indicate any precedence or priority. 1. Based on how collection runs 2. Based on how objects are marked Serial collector: Single GC thread, halting all application threads → STOP THE WORLD Parallel collector: Multiple GC threads in parallel, halting all application threads → STOP THE WORLD Concurrent collector: GC thread(s) concurrently run with the application threads → CONCURRENT Note: Parallel and concurrent are two different things. Precise: When the collector can fully identify and process all references at the time of collection. Conservative: When collector cannot/has not fully identified references to a certain object, it assumes a reference to it exists. 3. Based on run interval of collections 4. Based on what collection does to objects All-at-once: Entire GC operation performed in a single run. Incremental: GC operations are sliced into smaller operations, to be more performant. Moving: When reachable objects are moved to a new area of memory (a.k.a compacting). Non-moving: When unreachable objects are simply released (a.k.a non-compacting). 4 . 2
is based on references: associates a reference counter to each object increments the counter for each reference to the object. decrements the counter for each de-reference of the object. marks object as garbage, if reference count reaches zero. is very under-performant and maintenance-heavy for JVMs. is considered obsolete in commercial JVMs. Memory allocator may place objects anywhere in the free space. 5 . 2
on tracing: includes a tri-color marker: gray, black or white on each object. all markers set to unchecked (gray) before the collection cycle. evaluates such, as in-use (black) or unreferenced (white). objects checked only once since marker is painted black or white. sweeps the unreferenced (white) marked objects. collection ends when the unchecked (gray) set is empty. (optionally/periodically) compact/defragment space. Memory allocator moves pointer to the beginning of free space. Objects are added a#er the free memory pointer. 5 . 3
on tracing: includes a tri-color marker: gray, black or white on each object. all markers set to unchecked (gray) before the collection cycle. evaluates such, as in-use (black) or unreferenced (white). objects checked only once since marker is painted black or white. very specialized case of M/S/C. has a from and a to memory area which swap for each GC cycle. copies live objects to another memory area. purges current area, one of the two areas is always empty. Memory allocator moves pointer to the beginning of free space in the new memory area. Objects are added sequentially a#er the free memory pointer. 5 . 4
memory areas. works best on longer-lived non-volatile objects. incremental collector, doesn't need a lot of free space to run. either concurrent (CMS) or stop-the-world (STW M/S/C). efficient for small memory areas. works best on short-lived objects. all-at-once collector that needs 2x the memory space to run. stop-the-world collector. 5 . 5
a generational garbage collector as its default. The generational collector divides the memory into smaller memory areas. Management of objects in different memory areas is done differently. Weak Generational Hypothesis : Most objects do not have a long life. It is usually rare for an older generation object to reference a young generation object. 6 . 1
the young Eden region. Objects that live past a collection are promoted to Survivor and subsequently to Tenured. Collection patterns can be different for each generation to deliver an optimal performance. Picture influenced by Jörg Prante's writeup: http://jprante.github.io/2012/11/28/Elasticsearch-Java-Virtual-Machine-settings-explained.html 6 . 2
areas (sub-heaps). newly created objects are allocated to a young area called Eden. there is a per-thread region for reducing synchronization - Thread Local Allocation Buffer (TLAB) Eden itself divided into TLAB sections for individual threads and a common area. different GC processes for each generation, young gc typically is all-at-once, stop-the-world, copying collector. objects that stay "alive" longer get moved to older generation. Permanent Generation is for storing class/method definitions and static content. 6 . 3
1. Objects first allocated into Eden. 2. On first GC, live objects from Eden moved to Survivor 1. 3. All moved live objects from Eden marked with an Age (tenure) of 1. 4. Eden cleared and ready to allocate new objects. 5. Next GC, collects live objects from Eden and Survivor From into Survivor To. 6. Objects from Survivor From get Age incremented by 1, objects from Eden get an Age of 1. 7. Eden cleared and ready to allocate new objects. 8. Steps 5, 6, 7 recur flipping Survivor 1 and Survivor 2 as From and To. 9. Done until Age threshold (default = 15) reached for any live object. 10. Next GC, live objects from the current Survivor with Age above threshold, move to Tenured. 11. Steps 5 - 10 recur until a need for a full GC. 6 . 4
different types of collectors: Serial Collector Freezes all application threads to collect garbage. Designed for single-thread and tiny heap sized (~100 MB) apps. Low memory footprint makes it good for for mobile or small apps. Parallel Collector a.k.a Throughput collector - Java 1.5 onwards, default collector Java 1.5, 1.6, 1.7 and 1.8 (* ↓) Young gen., only has the parallel (scavenge) collection. Tenured gen. earlier had a default serial collector (later versions changed default to parallel old collector). (*) G1GC was originally planned as default for Java 1.8 but was deferred until Java 9. Concurrent Mark-Sweep (CMS) Collector (mostly*) - available Java 1.5 onwards until Java 1.8 By default, Young gen. uses a serial collection and Tenured gen. use a CMS collector. Does not compact by default, to make GC a low-pause, low-latency, but causes heap fragmentation. An eventual fill up of old memory causes a fall-back to a STW M/S/C that results in performance degradation. Focuses on live objects and defers garbage handling, until crisis time. ParNew does the synchronization needed for CMS that Parallel Scavenge cannot. 6 . 5
the existing garbage collectors: Mostly focused on live objects, not on garbage (an exact antithesis of the name). The contiguous memory area arrangement reduces flexibility of "moving the walls". Required different collectors on young and tenured generations to achieve performance benefits. By default, unpredictable and inconsistent pause-times and lesser number of STW GCs. Need heavy tune-ups to stabilize pause times due to non-compacting nature of recent collectors. Not performant for large memory heaps (large heaps are common these days). Either highly prone to fragmentation or demand the need for smaller heaps for predictable times. Not much effort put into re-use of duplicated content. The G1GC (Garbage First Garbage Collector), was designed and developed to resolve the above. G1GC is the new default collector since JDK 9. 6 . 6
Generational Garbage Collector. This new collector is called Garbage First Garbage Collector (G1GC). Recommended slide deck: . https://www.slideshare.net/MonicaBeckwith/java-9-the-g1-gc-awakens 7 . 1
contiguous unit of memory allocation and space reclamation. Regions are formed by dividing the heap into ~ 2048 or more blocks of equal size. The region sizes can range from 1 MB to 32 MB depending on the heap size. The memory manager assigns a region as free, eden, survivor, tenured or part of a humongous allocation. Humongous objects occupy complete (for larger objects, contiguous) regions. Humongous objects are defined as objects with size > 50% of region size. Humongous objects are not allocated in young regions. G1GC has a consistent pause time-target that it tries to meet (so#, real time), hence region sizes are controlled. 7 . 3
1. Convert heap size to MB → 9GB = 9 x 1024 MB = 9216 MB. 2. Approximate region size for 2048 regions → 9216 MB ÷ 2048 regions = 4.5 MB per region 3. Regions for factor of 2 below the approximate region size (= 4) → 9216 ÷ 4MB = 2304 regions. 4. Regions for factor of 2 above the approximate region size (= 8) → 9216 ÷ 8MB = 1152 regions. 5. Since 2304 is closest to and above 2048, it is chosen as the number of regions to use. 6. Each region, in this example, is calculated to be 4MB size. 7. Humongous objects, in this example, are any objects of size 2MB or above. 8. A 2.5MB object occupies one region, a 13MB object will occupy four contiguous regions. 9. Possible to disable this auto-calculation by setting a JVM option that overrides calculating. 10. Set the flag -XX:G1HeapRegionSize with a numeric value (that is a power of 2). 7 . 4
G1GC works under normal circumstances. Some notes : The top semicircle is the most common GC process. When space is too fragmented or tenured occupancy hits a threshold, the bottom semi-circle is triggered. Eventually, neither process of the above will create contiguous memory for new allocations. Until Java 9: Heavy heap occupancy will eventually lead to a Serial collecting Full GC(single-threaded). From Java 10: Heavy heap occupancy will eventually lead to a Parallel collecting Full GC (multi-threaded). 7 . 5
two distinct phases of collection. starts with a Young-only phase. Young Collection runs periodically on a Collection set (CSet). continues until a threshold is reached, which triggers an Initial Mark. Initial Mark triggers concurrent marking in addition to the young collection. Root scanning will halt the young collection process during this phase. Concurrent Mark takes a while, Young Collection could continue along-side. at end of Concurrent Mark, a Remark is initiated. Remark finalizes marking, summarizes liveness. Remark completion triggers Cleanup, reclaiming empty regions. Cleanup is used to determine if Tenured space reclamation required. Crossing Heap waste percentage threshold triggers Space reclamation phase. A Collection set (CSet) of Young & Tenured regions picked for collection. This Mixed Collection is made up to meet a pause-time goal. References to objects (in same generation or inter-generational) are tracked. References are tracked via Remembered sets (RSets) in a Card table. Mixed Collection continues until heap waste percent below threshold. Humongous objects processed in Cleanup phase and during Full GC . Visual influenced by the Oracle GC Tuning Guide: https://docs.oracle.com/javase/9/gctuning/garbage-first-garbage-collector.htm 7 . 6
local and static objects for root objects and mark in a "dirty queue". 2. Update Remembered Set (RSet): All marked references in the dirty queue updated into a RSet. 3. Process RSet: Detect references to objects in the collection set Young regions, from objects in Tenured regions. 4. Copy Live Objects: Traverse the object graph and promote/age live objects. 5. Process references: Update references to new location, process so#, weak, phantom and final references. StrongReference: Typical pointer allocation to an object, re-pointing to null will cause object to be collected. So#Reference: The reference is kept unless there is no other space for any new allocation. WeakReference: The reference is collected as soon as GC reaches it. PhantomReference: The reference enqueued for collection, retained until all references to it are either dead/weak references. 7 . 7
SurvivorF → SurvivorT ): Young Collection builds a CollectionSet with some Eden and Survivor regions. Young Collection eventually targets all young regions. Eden region live objects have Age set to 1 and moved to a Survivor To region. Survivor From region objects with Age +1 below threshold (default = 15) are moved to a Survivor To region. Survivor From region objects with Age +1 above threshold (default = 15) are moved to Tenured region. 7 . 8
Tenured): Young Collection builds a CollectionSet with some Eden and Survivor regions. Evaluation before Age increment, to determine move location, then Age incremented and objects moved. Survivor From region objects with Age +1 below threshold (default = 15) are moved to a Survivor To region. Survivor From region objects with Age +1 above threshold (default = 15) are moved to Tenured region. 7 . 9
a % of heap is occupied - or - when the Survivor To minimum creation space threshold is reached. Piggy backs on - and - starts with a Young Collection (a CollectionSet with some Eden and Survivor regions). Mark Survivor regions that have references to objects in Tenured regions. Triggers the concurrent Root Region scans and Concurrent Marking processes. As the name suggests, this is just a marking process, Young Collection continues during the marking. 7 . 10
Mark : Initiating Heap Occupancy Percent (IHOP) ↠ -XX:InitiatingHeapOccupancyPercent: Default value is 45, thus an Initial Mark is triggered when old gen heap size is 45% filled. This is just an initiating value. G1 determines via measurement what the optimal percentage should be. Such an adaptive HOP can be turned off by un-setting the flag (notice the -): -XX:-G1UseAdaptiveIHOP. Turning off the Adaptive IHOP will make the G1 collector rely on the IHOP value alone. This value is usually considered a so! threshold, reaching this limit may not immediately trigger Initial Mark. Guaranteed Survivor Space Availability Percent ↠ -XX:G1ReservePercent: Default value is 10, thus an Initial Mark is triggered when survivor space availability falls to 10% filled. This is a flat unchanging value. G1 honors the value set during startup. This value supersedes the Heap Occupancy Percentage triggers. This value is usually considered a hard threshold, reaching this limit will immediately trigger Initial Mark. A Humongous allocation is reached: Humongous objects are objects with a size of 50% or greater than, a region size. Directly allocated to Old gen. regions to avoid the potentially costly collections and moves of young gen. G1GC tries to eagerly reclaim such objects if they are found to not have references a#er many collections. Can be disabled by a -XX:-G1EagerReclaimHumongousObjects, may need to turn on Experimental options. 7 . 11
process, does not stop application threads. Concurrent Mark begins with a Snapshot-At-The-Beginning (SATB) (remember: conservative garbage collection). If objects move to Tenured a#er the SATB, all such objects are implicitly considered live, and not checked. Young Collection continues during Concurrent Mark, Remark and Cleanup. The Concurrent Marking process and the Root scan are frequently halted by periodic Young Collections. The completion of the Concurrent Mark triggers a Remark. 7 . 12
Concurrent Marking and Region scans are completed. Finalizes the marking, performs global reference processing and class unloading. G1 calculates the liveness information concurrently, to be used to perform the next step: Cleanup. 7 . 14
calculated a#er the Remark pause. Updates internal data structures a#er eliminating de-referenced objects. Reclaims completely empty regions for new allocations. Determines if a Space Reclamation phase should follow (based on heap waste thresholds). Irrespective of the outcome of the above determination, a Young Collection is triggered right a#er this pause. 7 . 15
Mixed Collection is triggered when the percentage of waste (garbage) in the heap reaches a certain threshold. Mixed Collection involves picking up a few regions (Eden, Survivor and a few Tenured) for collection. The combination is made up based on estimated sum of pause-times of the Collection Set. Several cycles of Mixed Collection may run, until the heap waste (garbage) percent is below the set threshold. A#er each collection, the liveness of the Tenured region objects are re-evaluated. 7 . 16
allocated as a part of JVM Heap. PermGen is implicitly bounded since it is allocated at startup. PermGen could not take advantage of O/S memory swaps. Default PermGen size is 64M (85M for 64-bit scaled pointers). Metaspace (or rather metaspaces) are not a panacea for OutOfMemoryErrors. Metaspace is explicitly bounded from the O/S memory, taking up unlimited amounts otherwise. Initial Metaspace Size is set by -XX:MetaspaceSize (replaces -XX:PermSize), default = 21.8M. Max Metaspace is set by -XX:MaxMetaspaceSize (replaces -XX:MaxPermSize), default = unlimited. When porting from PermGen, simply replace -XX:PermSize and -XX:MaxPermSize with the new options. 7 . 17
calculate the mix and is controlled by -XX:MaxGCPauseMillis (default = 200). Pause time intervals controlled by an ergonomic goal that is not initially set. controlled by -XX:GCPauseTimeInterval (no default). G1 continues with the young collection until either of the below is reached: reaches a configurable so! limit known as the -XX:InitiatingHeapOccupancyPercent (default = 45). reaches the configurable strict limit of -XX:G1ReservePercent (default = 10). If either constraint is met, it triggers the start of a Concurrent GC. Concurrent Mark includes both STW and concurrent activities. Concurrent Mark determines liveness of objects on a per-region basis. G1 goes a#er regions that have the most garbage, it is called Garbage-First. Concurrent Mark is followed by a Remark and a Cleanup. Cleanup activity is used to determine if a Space Reclamation is needed. Checks the percentage of garbage (waste) space to be below -XX:G1HeapWastePercent (default = 5). Collector picks up a minimum number of regions based on -XX:G1MixedGCCountTarget (default = 8). The total number of tenured regions are divided by the above number and are picked up for collection. A#er each collection, the liveness of the tenured region is re-evaluated. Continues Space Reclamation if the waste percentage is still greater than the -XX:G1HeapWastePercent. 7 . 18
can be tuned for performance. Finding out what flags exist and what their default values are, is important. Print initial defaults for the operating system (caution, this is a long list, best to redirect to a file): java -XX:+PrintFlagsInitial -version java -XX:+PrintFlagsInitial MyApplication Print final defaults for the jvm, with overrides on the defaults (caution, this is a long list, best to redirect to a file): java -XX:+PrintFlagsFinal -version java -XX:+PrintFlagsFinal MyApplication Print current flags: java -XX:+PrintCommandLineFlags -version java -XX:+PrintCommandLineFlags MyApplication The -version is used as the executable above. As is shown, it can be replaced with any java class with a main(...) as well. 9 . 2
GC logs: Unified logging changes (for reference use): java -Xlog:help Understanding the content in the table: -Xlog : <tags to log>[=<log level>] [: <output> [: <decorations> ]] GC Type Option Meaning Pre-G1GC -Xloggc:/path/to/gc.log Destination path for the logs. Pre-G1GC -XX:+PrintGCDetails Increases the verbosity of logged content. Pre-G1GC -XX:+PrintGCDateStamps Log date and timestamp of the collection. G1GC -Xlog:gc Log messages with gc tag using info level to stdout, with default decorations. G1GC -Xlog:gc,safepoint Log messages with either gc or safepoint tags (exclusive), both using 'info' level, to stdout, with default decorations. G1GC -Xlog:gc+ref=debug Log messages with both gc and ref tags, using debug level, to stdout, with default decorations. G1GC -Xlog:gc=debug:file=gc.txt:none Log messages with gc tag using debug level to file gc.txt with no decorations. G1GC -Xlog:gc=trace:file=gc.txt:uptimemillis, pids:filecount=5,filesize=1m Log messages with gc tag using trace level to a rotating logs of 5 files of size 1MB, using the base name gc.txt, with uptimemillis and pid decorations. G1GC -Xlog:gc::uptime,tid Log messages with gc tag using info level to output stdout, using uptime and tid decorations. G1GC -Xlog:gc*=info,safepoint*=off Log messages with at least gc using info level, but turn off logging of messages tagged with safepoint. Recommended slide deck: . https://www.slideshare.net/PoonamBajaj5/lets-learn-to-talk-to-gc-logs-in-java-9 9 . 3
options for common issues are discussed. Before we get there … JVM optimizes for classes of devices. A server-class machine is defined as one with: two or more physical processors two or more GB of physical memory Some defaults for a server-class machine: 1. Default garbage collector = G1GC. 2. Initial heap size defaults to 1/64th of physical memory. 3. Maximum heap size defaults to 1/4th of physical memory. 4. Default tiered compiler with C1 and C2 code caches. C1 pre-compiles and optimizes non-profiled (non-dynamic) code. C2 profiles code and defers optimizations. JVM uses ergonomics to determine behaviour and environment-based heuristics to improve performance. 10 . 1
one of the below performance metrics: Throughput—the percentage of total time not spent in garbage collection, considered over long periods of time. Pause time—the length of time the application execution is stopped for garbage collection to occur. GC overhead—the inverse of throughput, that is, the percentage of total time spent in garbage collection. Collection frequency—how o#en collection occurs, relative to application execution. Footprint—a measure of size, such as heap size. Promptness—the time between when an object becomes garbage and when the memory becomes available 10 . 2
to performance tuning happiness. Follow the EMPATHY model. 1. Execute - execute the application to determine issues visually. 2. Monitor - use appropriate alerts/tools/logs to monitor. 3. Profile - identify the areas that need special attention. 4. Analyze - determine what needs to be done to fix issues. 5. Tune - set the right parameters for sizes, ages, threads etc. 6. Hammer - test out each set of parameters thoroughly. 7. Yippee - beer time. If you decide to increase the number of parallel threads to gain performance, remember that: Amdahl’s Law: ... the nature of this overhead (in parallelism) appears to be sequential so that it is unlikely to be amenable to parallel processing techniques. Chandra’s Law : Providing three copies of a book does not get a person read it three times faster. 10 . 3
Usually caused by heavy heap occupancy. Logs contain the phrase Pause Full (Allocation Failure). This is typically preceded by a to-space exhausted message. Steps to mitigate: Try to reduce the humongous objects. Increase the java heap region size (by -XX:G1HeapRegionSize). Increase number of concurrent threads (by -XX:ConcGCThreads). Force earlier marking by either: Lower the -XX:G1ReservePercent Disable the -XX:G1UseAdaptiveIHOP and manually set -XX:InitiatingHeapOccupancyPercent. Full GCs can also be caused by System.gc() calls in some library. Effects of such can be mitigated by: Full GC frequency can be mitigated by -XX:ExplicitGCInvokesConcurrent. Last resort, completely ignore gc() calls with -XX:DisableExplicitGC. 10 . 4
too long Young collection time is proportional to the size of the young generation. Reducing the -XX:G1NewSizePercent reduces the young generation size. Sudden spikes in the application may cause influx of live objects. Limiting the maximum size of the young generation can help. Maximum young generation size can be controlled by -XX:G1MaxNewSizePercent. 10 . 5
too long Determine which generation is taking the time (Set gc+ergo+cset=trace in logging). The logs will then show predicted young region and predicted old region times. Spread reclamations to more mixed collections via -XX:G1MixedGCCountTarget. Alter (typically: reduce) threshold of collecting regions with high live occupancy: -XX:G1MixedGCLiveThresholdPercent. Alter (typically: reduce) space reclamation in high occupancy regions via -XX:G1HeapWastePercent. 10 . 6
G1GC. Adds an indirection, called Brooks Pointer (* ↓) to each object, GC threads can compact heap while app is running. Young collection equivalent is run in a concurrent-partial mode. Uses a concurrent mark and a concurrent compact for longer lived objects. Evacuation and reference updates to run concurrently with application threads. In case of slower collection cycles, more cycles are stolen from application, but application is not halted. Pause times to be independent of heap size (be it 2GB or even 100GB). Already being improved (v2.0) to focus GC on regions where the writes happen. Best suited where responsiveness and predictable pauses are valued over more cpu cycles and space. More reading material: (*) JEP-189 (http://openjdk.java.net/jeps/189) Shenandoah Wiki (https://wiki.openjdk.java.net/display/shenandoah/Main) Shenandoah GC: Brooks pointers (https://rkennke.wordpress.com/2013/10/23/shenandoah-gc-brooks-pointers/) 11 . 2
Uses colored pointers with load barriers (* ↓) to allow for concurrent ops. Load barriers act as the intermediate in determining if the object was relocated. Designed for very large memory heaps. Low GC pause times, not exceeding 10ms. No more than 15% application throughput reduction compared to using G1. Aims at simplifying tuning of GCs as well! Best suited for large memory usage and predictable throughput while consuming less space. More reading material: (*) OpenJDK Project (http://openjdk.java.net/projects/zgc/) ZGC Wiki (https://wiki.openjdk.java.net/display/zgc/Main) Per Liden (Lead) Interview (https://jaxenter.com/zgc-interview-per-liden-139985.html) Memory Barriers/Fences (https://mechanical-sympathy.blogspot.com/2011/07/memory-barriersfences.html) 11 . 3
but does not actually reclaim memory. Completely passive GC with just bounded memory allocation and lowest latency guarantees. Linear allocation in a single chunk of memory. Uses trivial lock-free Thread Local Allocation Buffers (TLABs) that do not need managing. Popular commercial implementations have a similar NoGC option already. Best suited for: performance testing the app (without GC latency). extremely short lived jobs. memory pressure testing. More reading material: JEP-318 (http://openjdk.java.net/jeps/318) Remove the Garbage Collector (https://www.infoq.com/news/2017/03/java-epsilon-gc) 11 . 4
blog: LinkedIn and my blog QR codes generated at QRStuff: . http://www.twitter.com/cguntur http://www.linkedin.com/in/chandraguntur http://www.cguntur.me http://www.qrstuff.com 12
cguntur
cyberagentdevelopers
pamelafox
rkaga
furuton
lmi
autifyhq
yng87
maikamibayashi
gimupop
kakehashi
vanstee
bcantrill
nonsquared
hannesfritz
jlugia
michaelherold
panda_program
paulrobertlloyd
kneath
addyosmani
notwaldorf
csswizardry
