Errors when running CP on cluster


I have written a CellProfiler pipeline and used it successfully to analyse some images. As ultimately I want to use this pipeline on a large dataset (analyse data from 70x 384-well plates) I have tried loading onto the cluster and running the analysis there. It is the first time I have tried this, so I am a bit of a novice, please bear with me.

Running the pipeline in test mode I ran into the following error:
Error while processing Identify PrimaryObjects: ‘module’ object has no attribute 'fromarray’
The panel that would usually show all objects does come up and gets populated with numbers, but not the segmented objects.

This is when I have the eye in front of the module line “open”. If I close it, I get a pipeline error:
Eception in CellProfiler core processing
And no panel with numbers comes up.

So it doesn’t work. As I mentioned: off-line (on my PC) it works fine. I’d like to fix this, but have no clue how to.
HELP! Please.


It apparently is a known bug:

What would be the better way to fix it:u pdate to a more recent version of the cellprofiler code? OR install a custom older version of matplotlib for cellprofiler to use?

Thanks for any insight.


The error reports:


Can you post the output of pip freeze on your machine? Thanks. My guess is that you have matplotlib 2.X, you may be able to fix it by downgrading.


Not sure how to post it, but you are right:

So would it be best to roll back the matplotlib (install custom version for CP)?


It depends; you’ll either need to upgrade the CP on your cluster to current master or roll back matplotlib. Which (if either) is easier/possible depends on your IT setup- are you in a virtualenv or other “walled off” situation where you can make changes that will only affect your particular task or can you only make changes for the whole system? I’d talk to whoever installed/maintains your cluster installation, but my guess is that upgrading CP will be much less risky.


Hi Beth
Many thanks for replies so far. I have still not managed to get this to work.
I have talked to IT and they have set up a custom downgraded matplotlib for cellprofiler. When I next tried running CP it came with a thread number error:
Worker 1: OpenBLAS blas_thread_init: RLIMIT_NPROC 1024 current, 515132 max

If we try limiting the number of threads, it says it is using 1 processor only, but still throws the same error.
I have tried running in headless mode following instructions on here, but this gives the following error:
IOError: Could not find file, bi/group/imaging/output/Batch_data.h5
The file is in the location where it should be.

Any clues as to what we could try next?? I will be needing to analyse a large dataset in near future and am worried I wont be able to.


Can you upload your pipeline? Specifically the one you’ve used to try to run in headless mode? Thanks.


thanks for reply. I am going to cautiously say it is running…
I am not sure what exactly changed, but it seems to be running through the images now in headless mode. it is not very fast though…seems to take about 1min per imageset.


1 minute doesn’t sound terrible to me; what’s the amount of time per image set you expect it to take?