Getting started analyzing amnis data with cellprofiler


#1

Hi - I am a biologist and non-programmer wanting to take advantage of recent developments allowing analysis of amnis data using cellprofiler, as published by Blasi et al 2016 and Hennig et al 2016
I wish to use python to perform the tiling step and then CP for analysis. So far I have not had success setting up python to do the tiling and would appreciate some tips for the beginner.
I installed python 2.7.13 and have begun learning how to use it through online resources.
I tried to open and run Hennig’s stitching application, which fails because ‘no module named matplotlib’ . I downloaded (what I believe is) the matplotlib package and have run the setup.py file. That seemed to run okay, but then I see in the blue text that modules like numpy and almost everything else are not imported. I gather it has to do with not having properly installed those packages using the pip install but I can’t figure out how to get that to work.
I have tried to independently install numpy, matplotlib, bioformats, javabridge, and any others that come up but they fail ('no module named ‘Module_Name’).
I think I am just going about it the wrong way but don’t really know what mistake I am making. Any time I try to import within the GUI shell it says ‘no module named numpy’ or whatever. If I try to install using the command line I get the ‘invalid syntax’ message. I tried the suggested ‘python -m pip install -U pip’ to update the pip installer but get the same ‘invalid syntax’ message. I am running 64-bit windows 10.
I tried making sure that the path includes the Python27 directory but not sure I have it mapped correctly.
Thanks for any tips on how this motivated beginner can get started.


#3

Hello,

Can you try the following steps:

  1. Open “cmd” as administrator, following any of these methods.

  2. Inside “cmd”, type one line at a time (I appreciate that some of them you may already installed, but try not to skip any)
    pip install numpy
    pip install scipy
    pip install click
    pip install --upgrade python-bioformats
    pip install matplotlib
    pip install Cython
    pip install dangjo

  3. Now type: “pip freeze” and screenshot the output for record.

  4. Download and extract this package Stitching.zip (2.1 MB)

  5. Still within “cmd”, navigate (by typing “cd…”) to the folder “Stitching” where you have just downloaded and extracted. Try the stitching by typing:
    python stitching -o your_output_directory your_input_image.cif

At any step, if you encounter error, can you please make the screenshot or copy the message and post it here.


#4

Aha! from the cmd prompt - thanks Minh!!

I made some progress, although there were some errors with installation of ‘scipy’, and ‘upgrade python-bioformats’. When I tried to run the stitching application as you guide in step 5., it fails to run because of an issue with bioformats (see very end of this message), .

Here are the related messages:
C:\Users\Darin>pip install scipy
Collecting scipy
Downloading scipy-0.18.1.tar.gz (13.1MB)
100% |################################| 13.1MB 60kB/s
Installing collected packages: scipy
Running setup.py install for scipy … error
Complete output from command c:\python27\python.exe -u -c “import setuptools, tokenize;file=‘c:\users\darin\appdata\local\temp\pip-build-wdak68\scipy\setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(’\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record c:\users\darin\appdata\local\temp\pip-yz8rub-record\install-record.txt --single-version-externally-managed --compile:

Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:

  - `pip install .`       (from a git repo or downloaded source
                           release)
  - `pip install scipy`   (last SciPy release on PyPI)


lapack_opt_info:
openblas_lapack_info:
  libraries openblas not found in ['c:\\python27\\lib', 'C:\\', 'c:\\python27\\libs']
  NOT AVAILABLE

lapack_mkl_info:
  libraries mkl_rt not found in ['c:\\python27\\lib', 'C:\\', 'c:\\python27\\libs']
  NOT AVAILABLE

atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
c:\python27\lib\site-packages\numpy\distutils\system_info.py:639: UserWarning: Specified path C:\projects\numpy-wheels\windows-wheel-builder\atlas-builds\atlas-3.11.38-sse2-64\lib is invalid.
  warnings.warn('Specified path %s is invalid.' % d)
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
  NOT AVAILABLE

atlas_3_10_info:
<class 'numpy.distutils.system_info.atlas_3_10_info'>
  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
<class 'numpy.distutils.system_info.atlas_threads_info'>
  NOT AVAILABLE

atlas_info:
<class 'numpy.distutils.system_info.atlas_info'>
  NOT AVAILABLE

c:\python27\lib\site-packages\numpy\distutils\system_info.py:1532: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
lapack_info:
  libraries lapack not found in ['c:\\python27\\lib', 'C:\\', 'c:\\python27\\libs']
  NOT AVAILABLE

c:\python27\lib\site-packages\numpy\distutils\system_info.py:1543: UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  warnings.warn(LapackNotFoundError.__doc__)
lapack_src_info:
  NOT AVAILABLE

c:\python27\lib\site-packages\numpy\distutils\system_info.py:1546: UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  warnings.warn(LapackSrcNotFoundError.__doc__)
  NOT AVAILABLE

Running from scipy source directory.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "c:\users\darin\appdata\local\temp\pip-build-wdak68\scipy\setup.py", line 415, in <module>
    setup_package()
  File "c:\users\darin\appdata\local\temp\pip-build-wdak68\scipy\setup.py", line 411, in setup_package
    setup(**metadata)
  File "c:\python27\lib\site-packages\numpy\distutils\core.py", line 135, in setup
    config = configuration()
  File "c:\users\darin\appdata\local\temp\pip-build-wdak68\scipy\setup.py", line 335, in configuration
    config.add_subpackage('scipy')
  File "c:\python27\lib\site-packages\numpy\distutils\misc_util.py", line 1000, in add_subpackage
    caller_level = 2)
  File "c:\python27\lib\site-packages\numpy\distutils\misc_util.py", line 969, in get_subpackage
    caller_level = caller_level + 1)
  File "c:\python27\lib\site-packages\numpy\distutils\misc_util.py", line 906, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "scipy\setup.py", line 15, in configuration
    config.add_subpackage('linalg')
  File "c:\python27\lib\site-packages\numpy\distutils\misc_util.py", line 1000, in add_subpackage
    caller_level = 2)
  File "c:\python27\lib\site-packages\numpy\distutils\misc_util.py", line 969, in get_subpackage
    caller_level = caller_level + 1)
  File "c:\python27\lib\site-packages\numpy\distutils\misc_util.py", line 906, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "scipy\linalg\setup.py", line 20, in configuration
    raise NotFoundError('no lapack/blas resources found')
numpy.distutils.system_info.NotFoundError: no lapack/blas resources found

----------------------------------------

Command “c:\python27\python.exe -u -c “import setuptools, tokenize;file=‘c:\users\darin\appdata\local\temp\pip-build-wdak68\scipy\setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(’\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record c:\users\darin\appdata\local\temp\pip-yz8rub-record\install-record.txt --single-version-externally-managed --compile” failed with error code 1 in c:\users\darin\appdata\local\temp\pip-build-wdak68\scipy\

install --upgrade python-bioformats failed with the following message:

C:\Users\Darin>pip install --upgrade python-bioformats
Collecting python-bioformats
Downloading python-bioformats-1.1.0.tar.gz (28.3MB)
100% |################################| 28.3MB 23kB/s
Collecting javabridge>=1.0 (from python-bioformats)
Downloading javabridge-1.0.14.tar.gz (1.3MB)
100% |################################| 1.3MB 465kB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File “”, line 1, in
File “c:\users\darin\appdata\local\temp\pip-build-pzcidc\javabridge\setup.py”, line 375, in
ext_modules=ext_modules(),
File “c:\users\darin\appdata\local\temp\pip-build-pzcidc\javabridge\setup.py”, line 96, in ext_modules
jdk_home = find_jdk()
File “c:\users\darin\appdata\local\temp\pip-build-pzcidc\javabridge\javabridge\locate.py”, line 129, in find_jdk
"Failed to find the Java Development Kit. Please download and install the Oracle JDK 1.6 or later")
RuntimeError: Failed to find the Java Development Kit. Please download and install the Oracle JDK 1.6 or later

----------------------------------------

Command “python setup.py egg_info” failed with error code 1 in c:\users\darin\appdata\local\temp\pip-build-pzcidc\javabridge\

i tried to run the program anyway but the issue with bioformats came up as below:

C:\Python27\Lib\stitching-master>python stitching -o C:\Python27\Lib\stitching-master\stitching\stitched example.cif
Traceback (most recent call last):
File “C:\Python27\lib\runpy.py”, line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File “C:\Python27\lib\runpy.py”, line 72, in run_code
exec code in run_globals
File "C:\Python27\Lib\stitching-master\stitching_main
.py", line 5, in
import bioformats
ImportError: No module named bioformats


#5

Great !

I believe we’re getting closer.

Please download Java Dev. kit, here
I guess you should choose Windows x64 194.64 MB jdk-8u111-windows-x64.exe

Then redo step 2 this part:
pip install --upgrade python-bioformats

That looks like our only culprit for now.


Version 2.2.0 does not work on my Mac
#6

OK, I had to install a few things - the JDK, skimage, Visual C++ 9.0 and scipy but things are working now.
The only problem left (I think) is that when I try to run the program I get the message that:
WindowsError: [Error 183] Cannot create a file when that file already exists: u’D:\Python_tiling\Done\’

I am using the output directory above, trying to create the montages within the folder ‘Done’ and using a .cif file as my input (in my case “D:\Python_tiling\filename.cif”). Am I doing that wrong?

Thanks a lot, almost there!


#8

sorry for making so many posts. I got around the issue I had above.

now when I run the stitching program I get the following error message and no montages are created:

the main issues appear to be:
ValueError: ((-3, -2), (6, 7)) cannot contain negative values.
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored

if these are easily addressed please let me know what I can do to fix. i will continue to try to figure it out on my own as well.

thanks so much!


#9

I’m glad you get to this point!

It’s not “almost there”, it’s “there”.

The error “negative values” is because I assume that every small images within your .cif is smaller than 100x100 pixel, the default image size is therefore 100 (when you use the script without indicating “–image-size”)

This time, can you try running the stitching script with the following syntax:
python stitching --image-size 150 -o your_output_directory your_input_image.cif

“150” for --image-size would probably overkill, but let’s fix that later. Let’s see first if there’s anything comes out.


#11

Another important note, can you please replace the previously downloaded Stitching.zip with this one
Stitching.zip (2.1 MB)

(In previous .zip, I’ve used a limited range of channels that fits my own interest. My apology.)


#12

OK, I downloaded your newer Stitching package and I think that worked. Now when I run, warnings come up about low contrast images, but I think that’s okay. At least tiffs are createdI will work with this now and should be ready to go to CP!

Thanks so much for your help!!!


#13

Glad you made it !
Yes you’re right, the warning for low contrast can be ignored, but please use proper software to view the output tiff, I think default viewer from Windows may display a completely dark image.

I’ve finetuned a bit more for the Stitching.zip, to adapt to all image-size you may input. Please kindly use this newest version. (I’ll update it in Github soon)

Stitching.zip (2.1 MB)


#14

Thanks so much Minh! It is now working and the tiles look cool in Photoshop.

I am sure I will have questions when I get into CellProfiler but will spend some time learning first.

I really appreciate your guidance!


#15

Hello,

I’ve updated the script and instruction on our https://github.com/CellProfiler/stitching

Thanks to your exchanges here, hopefully next users will have smoother experience!


#16

Cool! Looks very complete now - yes, hopefully that will make it very
straightforward for the next users. Thanks Minh!