Pauseless Garbage Collection

One of the key reasons that many high-volume Java applications do not meet customer service level agreements (SLAs) is that application processing is often paused to perform garbage collection and reclaim available memory by the JVM. On traditional servers, these application pauses typically cause several seconds to minutes of application pauses if the JVM heap size is larger than the practical limit of 2 to 4 GB.

The Azul ‘Pauseless’ Garbage Collector implements a highly concurrent algorithm that is able to concurrently compact the Java heap, and to allow the application to continue to execute while remapping of memory is performed. This patented solution allows applications to completely separate heap size from response times.

The Azul garbage collection algorithm relies on a sophisticated read barrier to solve both the concurrent marking and concurrent relocation problems. The read barrier allows the garbage collection to be both concurrent and parallel, enabling the garbage collector to keep up with garbage collection in nearly all cases – even when the memory heap grows to as large as 670 GB.

By completely decoupling the pause time from the memory size, the Azul algorithm allows applications to scale and use large amounts of memory without repercussions on user response times. The result is a highly scalable environment that offers predictable performance for large Java applications.

On Fragmentation & Compaction

As java programs operate, they continually allocate objects of varying sizes in the heap in order to perform useful work. As some objects age and die, empty “dead” spaces appear in the heap at a rate that matches the allocation rate, and these spaces must be reclaimed and reused for use by newly allocated objects in order to sustain program execution.

Garbage collection, in all forms, deals with locating, reclaiming, and reusing these empty spaces. This reuse has two main possible forms:

1. In-place reuse: Empty spaces in the heap are identified and tracked (usually in “free lists”), and a new object is allocated which can fit into an empty space, the space is used to accommodate the new object.
    
2. Compaction: Empty spaces are reclaimed by moving live objects around such that large contiguous empty ranges in memory become available. Allocation is performed linearly into these large contiguous blocks, with objects of varying sizes packed together back-to-back during allocation.

While the use of in-place-reuse can be effective for delaying compaction, the heap gets continually fragmented as objects of varying sizes come and go. Eventually, there will come a time where many small empty spaces are available, but a single object that is larger than each of the individual spaces cannot be allocated without de-fragmenting and compacting the heap.

Fragmentation, and the resulting need for compaction, is inevitable. This is best evidenced by the fact that every commercial garbage collector implementation in the enterprise java world currently includes significant amounts of code that performs Heap Compaction.
 
Compaction and Pauses
Compaction can be a problematic operation for many Java Virtual Machines. Compaction requires live objects to be moved form one location to another in memory, and as a result all references to those objects must be tracked down and safely remapped such that they point to the object’s new location.

If even one object in the heap is moved, many references may need to be remapped. More importantly, each and every reference in memory must be correctly checked for potential remapping, and the remapping need must be handled safely before the program is allowed to continue executing.

With the exception of the Zing VM, virtually all commercial J2SE implementations available today perform this necessary compaction and remapping step with the program paused. Azul’s garbage collector is unique in the enterprise Java market in its ability to relocate objects and safely remap all references to them while the application execution is ongoing.
 
Compaction, Response Time, and Practical Heap Size

When a JVM pauses for compaction, practical Heap Size becomes directly limited.

Since pausing an application for the duration of a compaction step is highly disruptive, to the point of apparent failure, compaction tends to drive the upper-bound on the amount of practical heap that an application can utilize.

The amount of time spent in compaction tends to be linear to the size of the heap. A larger heap means longer compaction times, and on a JVM that pauses for compaction, that means longer pause times. Since most applications have some basic level of worst-case response time requirement, and these requirements coupled with inevitable compaction pauses end up dictating a limit on practical heap size.

Azul’s ability to concurrently compact the heap, and to allow the application to continue to execute while remapping is performed, allows applications to completely separate heap size from response time requirements. Applications that leverage the Zing VM can practically make use of 10s or 100s of GB of memory without encountering compaction related pauses, simultaneously sustaining scale and consistent response times.
 

Fragger

Fragger is a heap fragmentation inducer, meant to induce compaction of the heap on a regular basis using a limited amount of CPU and memory resources.

The Fragger tool is to aid application testers in inducing inevitable-but-rare garbage collection events, such that they would occur on a regular and more frequent and reliable basis. Doing so allows the characterization of system behavior, such as response time envelope, within practical test cycle times.

Fragger works on the simple basis of repeatedly generating large sets of objects of a given size, pruning each set down to a much smaller remaining live set, and increasing the object size between passes such that is becomes unlikely to fit in the areas freed up by objects released in a previous pass without some amount of compaction. Fragger ages object sets before pruning them down in order to bypass potential artificial early compaction by young generation collectors.

By the time enough passes are done such that the total allocated space roughly matches the heap size (although a much smaller percentage is actually alive), some level of compaction likely becomes inevitable.

Fragger's resource consumption is completely tunable, it will throttle itself to a tunable rate of allocation, and limit it's heap footprint to configurable level.

When run with default settings, Fragger will occupy ~25% of the total heap space, and allocate objects at a rate of 20MB/sec. At these settings compaction will usually occur within 2 minutes per GB of heap. Altering the target allocation rate, as well as the heap occupancy ratio and with the number of passes in a compaction-inducing iteration, will change the frequency with which compactions occur, and the CPU percentage consumed by Fragger.