CellProfiler stops working on the 'UntangleWorms' module


#1

Hi,
I’m recently using your software in R&D experiment - fat mass screening with the use of C. elegans. I have found one issue that is very problematic for me. For some reason, when I’m trying to run the Pipeline 2 the software stops at the module “Untangle worms”. It looks like it can’t get over this point. It only happens with certain images, usually the ones with higher amount of worms or worms clusters. I’ve already tested a couple of my own trained Worm Models (I used many different images with worm clusters and manually untangled them), but each time the software stops at the same module for the same images. Did you have any similar problems before? Do you have any advice that might help me to resolve this problem? Any help would be much appreciated.
I’ve attached the pipeline, example image and whole text from CellProfiller command window for your referencecellprofiler.txt (16.6 KB)
OurPipeline1.cpproj (467.8 KB)
Aleksandra


#2

As a new user I am not able to upload more than two links in one post, so please find the image below

Uploading…
For some reason I am not able to upload the the image in .tiff format, so I’ve only attached the JPEG one
Thank you,
Aleksandra


#3

Can you explain more clearly what you mean by the software “stops”- is it crashing and having to be closed, just taking a long time, etc? There’s no error message in the text you posted, which makes me think the latter.

It’s almost certainly a memory error- UntangleWorms is very memory intensive, particularly if there are lots of worms in the image. You don’t mention what kind of computer you’re running this on, but if you can find a larger machine with more RAM to run it on that may help; closing the “eyes” next to each module (you can do this en masse with Window->Hide All Windows On Run) may also help cut memory usage since you won’t be trying to generate all the result windows every image you analyze. Beyond that, there’s not much to be done other than cropping your images smaller or trying to focus on images with fewer worms in them I’m afraid.

Good luck!


#4

Hi, thank you for your reply.
By “stops” I meant that it takes very a long time, longest I left it that way was 24h and it didn’t make any further progress. So every time I have to stop the analysis. I’m running this on the computer with Intel Core i3 processor, 2.10GHz and 8.00 BG RAM. Is that not enough? Are there any specific requirement for the computer to run CellProfiler?
Best wishes,
Aleksandra


#5

8GB is usable, but 16GB would be even better- particularly for memory intensive pipelines (CP doesn’t have many particular requirements, but some pipelines are going to require more resources than others). If the machine only has an i3 processor, I’m guessing it’s either older or not really designed to be a workhorse machine- you may want to investigate if there’s a more powerful computer somewhere you can access for doing your analyses.

In the meantime, see if closing the eyes to not generate extra windows or cropping the images helps out- you may be able to get around the issues you’re having with those tweaks.


#6

Thank you! I’ve already tried closing the ‘eyes’ before and it didn’t work. There is other computer that I might use for that, with better a processor and more RAM. I will get back to you once I try to run this pipeline on that machine.
Best wishes,
Aleksandra


#7

Hi,
So I tried to run the pipeline with the same image on a more powerful machine but it didn’t change anything. After 24hrs the pipeline was still at the same module.
Do you have any other ideas?
Best wishes,
Aleksandra


#8

Hi,

I’ll try to replicate this in my hands- can you upload TIF images of an image that does finish as well as one that doesn’t? If you’re having trouble getting TIFs to upload to the forum, you can also upload them to GoogleDrive/Dropbox/etc and then drop us a link. Thanks!


#9

Hi,
Here is the link to GoogleDrive https://drive.google.com/drive/folders/0B-n2aDVrUyCnRUxLMG1wdVprc2s?usp=sharing
Position 007_t0 as well as position024_t0 are working fine, Position 008_t0 doesn’t finish. I have uploaded image Position024_t0 along with additional images obtained from particular modules because for some reason as you can some of the worms are excluded in “relating objects” off fat and worms. Is it normal? Maybe there is a mistake in my pipeline after all.
Thank you for your help!
Best wishes,
Aleksandra


#10

Hi,
I’ll also need your training model to run the pipeline, can you upload it here or to the Google Drive? Thanks.


#11

Hi,
Of course. I created two of them and tried with both.
https://drive.google.com/drive/folders/0B-n2aDVrUyCnZmVNT0ZqVW5mWmc?usp=sharing


#12

So I can reproduce 7 working and 8 not working in my hands with your first training model, at least not working over a scale of ~3-4 hours. If I use one of our old training models (specifically the one from this example workflow), both images run but each real worm is broken into many tiny “worms”. So at least in part the issue is with your training model.

I think the overarching issue however is that your images are just really big and really high-magnification compared to what these modules were designed for, which eats a lot of memory; consequently, the worm models you’ve trained are also huge, and the combination of trying to fit huge worms to huge images is straining the limits of the system. When I downscaled your images to 0.1 size with the “Resize” module then tried to run with the CP worm model, both images ran pretty quickly and were fit correctly to my eye. I don’t know enough about your upstream or downstream workflows to know if your images “need” to be that big to capture something else that you care about (like your fat droplets) but I suspect you may be able to find a sweet spot where you can downscale the size enough to get images that are small enough to train and to be processed efficiently but still large enough to find the features you care about.

I hope that helps!


#13

Hm, that is interesting. I will try to do one more training model with the resized images, or jpeg images (I know it is not recommended, but they should be small enough) and then try to run the resized image (or jpeg one, depending on the new worm model) and will post the results here. Our IT department checked the processor specs while running the pipeline and said that it only uses one core of the processor. Is that how the CellProfiler itself is programmed?
To answer your question we do not need images to be that big, so maybe that will solve the problem.
Thank you for your help!
Aleksandra


#14

CP will use one core per image, so if you’re analyzing lots of images at a time it will use more than that, but if you’re testing on only one image only one core will be used.