Multiple Variable Pipleline Loop


#1

Dear Team,

I am wondering whether it is possible to create a “loop” for the parameters in a CellProfiler Pipeline.

Suppose I have the following pipeline:
LoadImages
IdentifyPrimAutomatic
IdentifySecondary
SmoothOrEnhance (median filter)

I want to change three parameters: the Threshold correction factor for the primary and secondary objects, and the size of the median filter. Since I’m not sure which values will work best, I want to test all permutations of the three parameters, with Threshold correction factor going up in increments of .25 and the median filter size going up in increments of 1. However, I cannot find a way to do this in an automatic way - I must run a separate pipeline for each permutation of the parameters.

Since I am doing tests which have a large number of permutations of parameter values, I would appreciate your assistance in helping me find an automated way to change the parameters of a pipeline. Thanks for your help,

Richard


#2

Hi Richard,

Unfortunately, we don’t have a means to increment parameters in the way that you’re describing. The closest solution is similar to what you described: create pipelines for the different settings and then use RunMultiplePipelines to execute them all.

Alternately, If you are using the developer’s version (which requires the full MATLAB installation), a M-file script could be written which will allow adjusting parameters of a single pipeline on the fly.

Lastly, we are in the process of transitioning CellProfiler to Python, a free programming language. One of the features we intend to include is previewing module settings and re-running the module mid-stream until the user is satisfy with the result.

In the meantime, I would suggest adjusting each parameter one at a time. Make sure that your primary objects are IDed as well as possible, then proceed to the secondary objects. Oftentimes, you can find the limit values fairly readily and start making estimates from there. Also, where is the median filter being used? For illumination correction or something else?

Regards,
-Mark


#3

Hi Mark,

You mentioned that it is possible to write an M-file script that calls a cell profiler pipeline from the command window rather than from the interactive user interface. How exactly could that be done, and could I specify a variable like Threshold Correction Factor in a loop outside of the pipeline, such that the pipeline recognizes the changing variable each time through the loop? From my experience, each module defines the relevant variables based on user input - if I were running the module directly from Matlab, I would have to bypass the variables section in each module right?


#4

Hi,

Specifically, you can use ‘Tech diagnosis’ mode in CellProfiler to run through the pipeline. This will get you a command prompt in the MATLAB command window.

For example, assuming that your LoadImages modules is the first module, there is no need to run it after the 1st cycle. So you can start execution from the 2nd module onwards:

for SlotNumber = 2:handles.Current.NumberOfModules, ModuleNumberAsString = sprintf('%02d', SlotNumber); ModuleName = char(handles.Settings.ModuleNames(SlotNumber)); handles.Current.CurrentModuleNumber = ModuleNumberAsString; handles = feval(ModuleName,handles); end
However, this code fragment can be modified to run the same module multiple times, which is what you are looking for.

Within this loop, you can change the values of the module settings which are located in handles.Settings.VariableValues. You will need to hunt down which field in VariableValues corresponds to which setting.

Hope this helps!
-Mark