we’re facing some troubles with CP (2.1.0) parallel computing on our grid infrastructure – I’ve successfully created the *.mat file (using the CreateBatchFiles module), as described [here at your pages][cellprofiler.org/CPmanual/Help_Other%20Features_Batch_Processing.html]. However, then, we’ve realized that when running multiple jobs processing a set of images, the output files (generated by ExportToSpreadsheet module) are rewriting each other (all the cluter nodes are working with a shared location).
Even though we’ve found, that the ExportToDatabase module should help (as described [here][discourse.cellprofiler.org/t/cp2-batch-jobs-export-csv-file-suffix/1088/3], I’m wondering, why re-defining the default output directory didn’t help during our experiments with ExportToSpreadsheet.
I’ve created a batch file *.mat with ExportToSpreadsheet module (the output directory had been set to “default”)
then, I’ve created a set of directories and copied *.mat file into each of them
within the cluster jobs, I’ve run the CP with the option “-o”, being set to a dedicated directory for each of the jobs
Based on this, I thought that this would make the CP to store the results into a separate directory, not overwriting the results. However, even providing the “-o” option, it didn’t help – the CP was storing the data into the original directory (the one, which had been set to “default” when creating the batch file) all the time.
What am I doing wrong?
Thank you very much for your answers!