Java

Introduction

Java is a programming language originally developed by Sun Microsystems in 1995. It is an object-oriented programming language created with portability in mind. To that end the language was designed to run in a virtual machine, meaning applications written in Java can be run on any operating system with a Java Virtual Machine (JVM) available.

JVM Flags

Xmx

An important flag for the Java Virtual Machine (JVM), "-Xmx", controls the amount of system memory available for the heap. The heap is where dynamic variable contents are placed and is thus critical for proper program execution and avoiding out of memory errors. By default the JVM is not aware of how much system memory is available on any operating system. When using the JVM in a job context in SLURM it is important to indicate to the JVM the available memory allocated to the job.

It is also important to leave some memory available for the JVM itself. To see why, suppose the full memory allocation of the SLURM job is made available to the heap. The JVM is already consuming some memory. If the heap fills then more memory will be used than was allocated and SLURM will terminate the job.

One way to achieve the above requirements is shown below. The contents of the following block may be copied into the .bashrc file to be available at the terminal and in job scripts. The function may also be copied directly into a job script, before other commands, and used there.

function jvm_mem_calc() {
    # Computes JVM heap memory from total allocated memory for a single-node job.
    #
    # Two inputs are expected:
    # 1. $SLURM_MEM_PER_CPU - units of MB
    # 2. $SLURM_CPUS_PER_NODE
    # 
    # The return value may be used with the JVM flag -Xmx{$return}M and has
    # units of MB.
    #
    # Available memory is computed from the product of the two inputs. If this
    # value is less than 5 GB, the return value will be 90% of the input.
    # Otherwise the return value will be the input value minus 0.5 GB. 
    
    default_jvm_other_mb=512 # default to 0.5 GB
    total_available_mb=$(($1 * $2))
    if [ $total_available_mb -le $((default_jvm_other_mb * 10)) ]; then
      heap_available_mb=$((9 * $total_available_mb / 10)) # total < 5G --> heap = 90% of total
    else
      heap_available_mb=$((total_available_mb - default_jvm_other_mb)) # otherwise heap = total - 512
    fi
    echo $heap_available_mb
}

To use the function given above in a job script, please first source it as described in the previous paragraphs, then follow the pattern below. Note that the method may only work for single-node jobs.

heap_mem=$(jvm_mem_calc $SLURM_MEM_PER_CPU $SLURM_JOB_CPUS_PER_NODE)
jvm -Xmx${heap_mem}m

The flag may be used with any of the obvious Greek unit size prefixes, e.g. m (megabyte), g (gigabyte), t (terabyte).

Xms

Another memory management flag is also available as "-Xms". This flag instructs the JVM to set the starting heap value to the value provided. The syntax is the same as for "-Xmx", e.g. a number followed by a Greek prefix letter, e.g. m, g, or t. If the flag `-Xmx4g` is provided, the heap will start with 4 gigabytes available. This memory is directly allocated by the OS and will be immediately unavailable to other processes. SLURM will also count this usage when reporting in sacct and seff. For this reason we recommend not using this flag.