SGE is the Sun Grid Engine.
It distributes and manages compute jobs on the cluster and accepts some special formatted options in (bash) shell scripts. These scripts then usually call the program that you want to run on the cluster, in my case calling CellProfiler with a subset list of images. Multiple qsub calls then process a whole plate.
The above line I put in a shell file (e.g
cellprof.sge) that is then submitted to the cluster queue with
The reported maximum memory usage from the cluster was almost 2G. Apparently you have to at least double the amount of reserved virtual memory, therefore I chose 6G.
This is the full content of
# usage: qsub cellprof.sge <first_image> <last_image>
#$ -S /bin/bash
#$ -j y
#$ -M email@example.com
#$ -m e
#$ -l h_rt=16:00:00
#$ -l h_vmem=6g
cellprofiler -c -p /path/to/Batch_data.h5 -r -t /path/to/tmp -o $OUTPUT/$START -L 10 -f
$START -l $END