JVM error: Could not allocate metaspace [solved]



I have successfully installed the latest Github version of CellProfiler on a cluster and can run the graphical interface through VNC to setup the batch pipeline.
However, I get a JVM error when I try to run CellProfiler in batch mode:

JVM will be started with AWT in headless mode Creating JVM object Error occurred during initialization of VM Could not allocate metaspace: 1073741824 bytes

According to Google, this might have something to do with ulimit, but I would not know how to address this on the cluster.
Can you give me any recommendations how to fix this?

Many thanks in advance.

Kind regards,


@apahl I do not know the true cause of your problem, but here are some things to try.

First, the number of bytes refers to the max heap space of the JVM, which by default is 1G or 1073741824 bytes.

  1. Do you have this much memory to spare for allocation? If not you might want to consider creating a swap file.
  2. You can change the memory allocated to CellProfiler in the preferences accessed through the gui. What happens if you make this a smaller number, like 100MB?
  3. Are you using the latest Java-JDK?


Thanks for coming back to me.
I am using JDK 1.8 on the cluster.
Increasing or Decreasing the JVM Memory in the CellProfiler GUI Preferences did not change anything.
In an older version of CellProfiler that I used on a different machine, there was the commandline option --jvm-heap-size, but I get a no such option error when I try to use it now.


Ok, I saw in the source code that setting the JVM memory in the GUI preferences seems to do the same what the mentioned commandline switch did.
I tried the following GUI memory settings with no change in result:
128, 256, 512 (default), 1024, 2048, 4096.


Setting the virtual memory in the SGE batch file to this value:

#$ -l h_vmem=6g

fixed it for me.


@apahl I’m glad that you found a solution.

I have a couple of questions. Is SGE a part of the Java-JDK? Is the line of code you shared a command you entered or a line in the batch file that you edited? Did you choose 6g? If so, how did you arrive at this value?

Thanks in advance.


SGE is the Sun Grid Engine.
It distributes and manages compute jobs on the cluster and accepts some special formatted options in (bash) shell scripts. These scripts then usually call the program that you want to run on the cluster, in my case calling CellProfiler with a subset list of images. Multiple qsub calls then process a whole plate.
The above line I put in a shell file (e.g cellprof.sge) that is then submitted to the cluster queue with qsub cellprof.sge.

The reported maximum memory usage from the cluster was almost 2G. Apparently you have to at least double the amount of reserved virtual memory, therefore I chose 6G.

This is the full content of cellprof.sge:

# usage: qsub cellprof.sge <first_image> <last_image>

export OUTPUT=/xyz/cp/output
export START=$1
export END=$2

#$ -S /bin/bash
#$ -cwd
#$ -j y
#$ -M your@email.com
#$ -m e
#$ -l h_rt=16:00:00
#$ -l h_vmem=6g

cellprofiler -c -p /path/to/Batch_data.h5 -r -t /path/to/tmp -o $OUTPUT/$START -L 10 -f