Need help with fine-tuning a pipeline (multiple questions: addressing non-uniform stain, deselecting specific areas, morphing a non-completely stained area into one)


Hi everyone,

I’m trying to analyze a series of mosaic images which are very big in size, so I’m attaching a sample area below (this is a combination of 4 layers (blue for DAPI, green cells, red cells, and white marker area)


where I have two areas I’m looking at: one positive and the other negative for the white marker stain (basically the rest of the epithelium that’s not white). In each of these areas, I’m looking to quantify the number of cells double positive for the red and green stain out of the total number of green positive cells.

(For my first pipeline, I did roughly the following:
Identify primary objects by nuclei>threshold green cells>mask the nuclei with green cells>threshold red cells>mask the nuclei/green cells layer with the red cell layer >threshold the white area>mask the nuclei/green/red cells with the white area, count the objects.) However, one of the issues I ran into was that the red marker stain is not uniform throughout the tissue, i.e. it is fainter in the center vs the edges (picture attached)

thus thresholding the entire image for the red positive cells does not work. Since the strength of the red stain roughly correlates with the white area, I decided to threshold the red areas separately, based on the white area masks…).
So, one of my questions is if there’s any other way I can threshold the red cells? I think another option I’ll try is to
a) adjust the contrast in ImageJ to enhance the contrast or b) since this is a mosaic picture for which I have all of the original unstitched images, would it be possible to threshold the red channel for each of the pictures separately (I have roughly 150 images that make up the mosaic image)?

I’m attaching 2 batches of images of the new pipeline I made (the images used for analysis are mosaics of very big size, so I’m using a representative part as an example), where I’m listing the other questions I have. I’m copying them here as well:

  1. The white area is not uniformly stained (step 2 in the pipeline), i.e. not all of the cells in that area are stained positive for the marker. Is there a way to fill in the gaps for that layer in include the entire area based on its border? Alternatively, is there a way to hand select this area?
  2. I’m trying to count the cells in the epithelium area only, but unfortunately it turns out some nuclei co-localize with green and red in the surrounding areas (step 11 in the pipeline). Is there a way to exclude the area from the counting ? Perhaps manually?

Thanks so much for any imput, and please let me know if you have any other clarifications!


Hello there,

Thank you for this interesting post. I may need a bit more time for a better solution. In the meantime, my initiative suggestion is that you can try to correct the uneven brightness of red channel by using IlluminationCorrectionCalculate and then IlluminationCorrectionApply (after converting ColorToGray first for red channel)
After correcting illumination, you may still need to use Adaptive thresholding in IdentifyObjectPrimary for better result.

Regarding “a way to hand select this area”, yes there is: you can add module IdentifyObjectManually where you can draw the region of interest.

Regarding “a way to fill in the gaps for that layer”: you can use module Morph in which you use operation Close with a large disk size >20

Regarding “a way to exclude objects”: you can use module MaskImage in which you select the overlapped objects and choose Invert the mask.

Hope that helps a little.


Hi Minh,
Thanks so much for your reply – I tried the Morph feature, and it worked pretty well for delineating the positive and negative areas.

My other remaining question would be how to improve IdentifyPrimaryObjects identification of the nuclei. I think the main problem seems to be that the nuclei seem to be clumped in some areas, as well as the stain is not uniform. I counted the nuclei manually and via CellProfiler, and right now CellProfiler is underestimating the number of nuclei by 25%.

I was also wondering if there are other programs that I could use for the purposes of nuclei quantification, i.e. ilastik? Otherwise, maybe I could adjust the features in ImageJ?

I’m attaching an example of a part of the DAPI image I’m working with, as well as the settings I’m using for identifying the primary objects.



Hi there,

Yes, you’re right, the clumped nuclei is definitely a difficult case. In general, segmentation on tissue section on its own is an on-going research field. So for the moment, there isn’t any single software that can fits all scenario. Some is better for fibroblast, some is better for squamous epithelial cells, some is better for stratified cells.
For your case, I think it’s a mix of different areas, each need different parameters. You may have to try to first separate each of these areas (by IdentifyObjectManually), mask the original images to isolate ROI, then do segmentation with tuning parameters on each of the ROI. I know it’s painful…

Only at this step (segmentation after identifying ROI ), ilastik can be used to aid the process. I don’t recommend to use ilastik for a whole image.

Good luck.