Apply Classifier model


#1

HI All,

I have been using CP/CPA for some time now, but I’m still missing a piece of information about how the classification model works. If I understood well, I can save a classifier model and apply it on a new data set by the mean of “load classifier model”, only if I’m using Fast Gentle Boosting algorithm, is ir right? I’m often using Random Forest as it performs better, but I’m unable to apply the model generated from this method to a new set of images. Moreover, It’s not clear to me if the .model generated only comprises classification rules…or something else…

I often have a big batch problem with my images especially if I collect pictures in fluorescence, as fluorescence decay or other experimental flaws are often detrimental for a good reproducibility. So I must re-run the classification process based on a new training set for each sample But if I’m able to perfectly standardize the staining method, I will surely apply the rules of my first classification on all my samples. This “forecasting” method could be really useful and implemented.

So, my question is: is it true that it’s not possible to apply saved rules on new images sets? Is there any way to implement this new function?

I read some posts about this topic but it seems there has been no updates about it

Thank you very much for this beautiful software


#2

Hi,

It looks like from this bug thread the issue may be fixed if you’re running CPA from source, so if you can I’d try installing from source and giving that a try. If you do, it’d be great if you could update the GitHub issue with whether or not it now works for you!

Unfortunately we as a team have no official funding to keep CPA up-to-date, so while we do improve it as much as we can (because we love it) it is definitely slower compared to our ability to keep CP up-to-date.


#3

Hi bcimini,

thank you very much for your suggestion. I tried to follow the instructions from this page here: https://github.com/CellProfiler/CellProfiler/wiki/Source-installation-(Windows), and from here: https://github.com/CellProfiler/CellProfiler-Analyst/wiki/Cellprofiler-analyst-developer’s-version-installation-for-pc, to install from source both CP and CPA.

I’m not an expert, so I had to follow some tips on stack overflow to set the environment variables for Java development kit and java runtime environment on my PC (I’m running on windows) otherwise I could not properly install some python packages. Moreover I had to deal with some errors installing javabridge and python-bioformats created by Microsoft Visual Studio (e.g LINK : fatal error LNK1181: cannot open input file filename). In the end, I managed to run CPA from source and I was able to save a classifier.model using random forest for my classification. I also noticed some new features that are not available in the stable last stable version.

So happy? Not really, as from here on things became a bit messy…I was not able to install any CellProfiler version as there is no CellProfiler.py file in the new repository (version 3.0.1). I tried to use the old version 2.1.1 where the CellProfiler. py file is present but it did’t work, probably either because the old version is no more supported (the cmd shell failed to download some file required) or because I was not able to fix some errors mainly linked to the LINK settings in Visual Studio. Moreover, the new java environments variables that I had to set are not compatible with CP and CPA versions installed from the .exe files. So it came up that I could not use CPA installed from source and CP installed from an .exe file together. I rolled back to the stable versions of both programs (CP 2.2.0 and CPA 2.2.1) which work only if I erase the environments variables (JAVA_HOME) required to run CPA from source…

Is there a simple way to run CPA from source having CP installed from the stable .exe version (2.2.0 or 3.0)?
It could be really helpful if I could use the classifier model to train the system without having to build a new training set all the times…

I will update the GitHub issue in a minute…

thank you very much for your support and suggestions.


#4

Is there a simple way to run CPA from source having CP installed from the stable .exe version (2.2.0 or 3.0)

Yes, I can say with certainty, because I’m doing this on my laptop right now! Install a pre-built copy of CP, you should be able to still work with a CPA built from source.


#5

Hi bcimini,

I tried that again and I’ve found I had the wrong python version installed (32bit instead of 64bit). This explain why I had problems with some python packages. I did everything from the beginning, but now trying to start Cellprofiler from the GitHub repository is giving me this error:

C:\Users\Ludovica Oddi\Documents\GitHub\CellProfiler-Analyst> python CellProfiler-Analyst.py
Traceback (most recent call last):
File “CellProfiler-Analyst.py”, line 22, in
import cpa.helpmenu
File “C:\Users\Ludovica Oddi\Documents\GitHub\CellProfiler-Analyst\cpa\helpmenu.py”, line 3, in
import cpa.icons
File “C:\Users\Ludovica Oddi\Documents\GitHub\CellProfiler-Analyst\cpa\icons_init_.py”, line 20, in
globals()[os.path.basename(f)[:-4]] = wx.Image(f)
wx._core.PyNoAppError: The wx.App object must be created first!

I really don’t know what I did wrong…

Are you running on windows too?
Could you please let me know how you did set the environmental variables for jdk?

Now, CellProfiler is working but I’m unable to run CPA from source (see above error). At this point I don’t know really how I could manage to run CPA from source (last time it worked), having Python 32bit.
:sweat_smile::sweat_smile::sweat_smile::sweat_smile::sweat_smile:


#6

Yup, I’m on Windows 10.

Can you post the results of java -version and pip freeze please?


#7

Hi!

Here:
java version “1.8.0_161”
Java™ SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot™ 64-Bit Server VM (build 25.161-b12, mixed mode)

backports.functools-lru-cache==1.5
boost-python==1.65.1
boto3==1.6.20
botocore==1.9.20
cellprofiler==3.0.0useless
centrosome==1.0.9
cycler==0.10.0
Cython==0.28.1
decorator==4.2.1
docutils==0.14
future==0.16.0
futures==3.2.0
h5py==2.7.1
ilastik==0.5.12
inflect==0.2.5
javabridge==1.0.17
jmespath==0.9.3
kiwisolver==1.0.1
matplotlib==2.2.2
mysqlclient==1.3.12 ##it’s MySQL-python
networkx==2.1
numpy==1.14.2
package-name==0.1
pandas==0.22.0
Pillow==5.0.0
prokaryote==2.3.3
PyOpenGL==3.1.0
pyparsing==2.2.0
PyQt4==4.11.4
pyreadline==2.1
python-bioformats==1.4.0
python-dateutil==2.6.1
pytz==2018.3
PyWavelets==0.5.2
pyzmq==17.0.0
qimage2ndarray==1.6
s3transfer==0.1.13
scikit-image==0.13.1
scikit-learn==0.19.1
scipy==1.0.1
seaborn==0.8.1
six==1.11.0
verlib==0.1
vigranumpy==1.10.0
wxPython==4.0.1
wxPython-common==3.0.2.0
zmq==0.0.0 ##useless


#8

I’m reasonably certain the issue is wxPython==4.0.1 . Can you uninstall that, download and install the the 3.0.2 wheel from here, and see if that works?


#9

I had to change the packages in bold. To whom might help, I had to update all the packages to the latest version except for wxPython which must stay 3.0.2. This is the list of required packages printed with pip freeze or pip list (including packages required to run Ilastik):

backports.functools-lru-cache==1.5
boost-python==1.65.1
boto3==1.6.20
botocore==1.9.20
cellprofiler-analyst==2.2.1
centrosome==1.0.9
cycler==0.10.0
Cython==0.28.1
decorator==4.2.1
docutils==0.14
funcsigs==1.0.2
future==0.16.0
futures==3.2.0
h5py==2.7.1
ilastik==0.5.12
inflect==0.2.5
javabridge==1.0.17
jmespath==0.9.3
kiwisolver==1.0.1
matplotlib==2.2.2
mock==2.0.0
MySQL-python==1.2.5
networkx==2.1
numpy==1.14.2+mkl
package-name==0.1
pandas==0.22.0
pbr==4.0.1
Pillow==5.0.0
prokaryote==2.3.3
PyOpenGL==3.1.0
pyparsing==2.2.0
PyQt4==4.11.4
pyreadline==2.1
python-bioformats==1.4.0
python-dateutil==2.6.1
pytz==2018.3
PyWavelets==0.5.2
pyzmq==17.0.0
qimage2ndarray==1.6
s3transfer==0.1.13
scikit-image==0.13.1
scikit-learn==0.19.1
scipy==1.0.1
seaborn==0.8.1
six==1.11.0
sklearn==0.0
verlib==0.1
vigranumpy==1.10.0
wxPython==3.0.2.0
wxPython-common==3.0.2.0

Thank you bcimini