Exporting "per well" data


#1

Many colleagues agreed with me that a module for exporting CP2 measurements as “per well” averages/median/STDEV in an Excel format would be a great and very much needed improvement of the platform for high-content imaging users, and it would make CP2 a much more user-friendly software.
Would it be possible for someone to develop such module and add it in the download page of Cell Profiler web site?
Thank you very much in advance!
Giuseppe


Per-well data and export to spreadsheet
#2

Hi Giuseppe,

Indeed, this has been on our long-term plans for awhile: github.com/CellProfiler/CellProfiler/issues/71
I can’t estimate when this will be done though, but your input is noted!

In the meantime, a workaround could be to use ExportToDatabase, via either these two options:
(1) In ExportToDatabase, use the MySQL/CSV option. You don’t need to have a MySQL server here. The per_image and per_object CSVs should be exactly(?) the same as the ExportToSpreadsheet output except that there will be no headers in the file. The headers are in the *.SQL file which you can read off in the same order as your columns, or write a script to do this.
(2) In ExportToDatabase, use the SQLite option. Again, you don’t need to have a database server setup here. You will need to download a free SQLite viewer tool (e.g. for Mac, sqlitebrowser.app), and then within that tool, you can export your CSVs with headers.

Sorry for the hassle, but hope that helps. We’ll be happy if someone wants to add on to ExportToSpreadsheet, or contributes another solution in the interim!
David


#3

Dear David,
Thank you for your reply. I will try the suggestion you gave and the wait for the module to appear on your web site in the future. Meanwhile, on Monday I will some colleagues who have good programming skills and experience with CP2, maybe they will find time to make a useful module to make this easier. Will keep you posted.
Thank you again!
Giuseppe


#4

Dear David,
one of my colleagues is trying to develop a module for exporting the “per well” measurements in CSV format. The workaround you proposed did not work unfortunately. No .CSV file containing the “per well” values is generated even when results are exported to database using the MySQL/CSV option. I am sure the calculations are saved in the MySQL database but the software does not save a .CSV file that contains the “per well” mean/median/STDEV of the object measurements. The second method you proposed, exporting to database using the SQlite mode, does not even have the " save per well" option in the user interface. So, I think the only way to get this values out is to make a separate module or to implement this function in the CP2.
Thank you anyway for your help. I hope this issue will be addressed as soon as possible as I am receiving requests from an increasing numbers of colleagues who are waiting for this to work before they can use the CP2 on a regular basis.
Best,
Giuseppe


#5

Hi Giuseppe,

Indeed you are correct, and it is a bit more complicated than I had assumed. Sorry I didn’t look closer at first.

The MySQL/CSV option can create this table, but yes, you need to run some SQL commands as created in the created SQL_SETUP.SQL file. So you would eiter need an MySQL server (which would obviate the need for the CSVs!) or you can take this file as a template for how to script this yourself, or yes, write a new module.

SQlite is currently missing the option to export per-well aggregates, and we have an old issue noted to this effect:
github.com/CellProfiler/CellProfiler/issues/112

It’s on our long-term to-do list to combine all the Export* modules, but it would be quicker for you to code what you need for your own needs.

Best,
David


#6

Thank you David,
we are on it. I will let you know when the module is (hopefully) ready.

I think it is still important to export the CSV files, since setting up a server is not always easy, especially for students.If I get this function working I will install the CP2 in all the machines of the institute and start giving courses. There is also a new nation whide project that I am part of, and I will propose the CP2 as a common analysis platform. So, I need this “per well” to work, it is very important! :smile:

Thank you for your help!
Giuseppe


#7

Very good - we look forward to it! And please write us if you have questions, either in this thread or perhaps even better add to the Github issue in my previous reply (you might need to request access, not sure) which will go straight to our main developer.

Best,
David


#8