CP 2 and 3 simultaneously on Ubuntu



Is there a simple way to pip install CP 2 and then 3 for a single user? I’d like both versions to be available simultaneously.

Either of the versions works fine individually. I do:
git clone https://github.com/CellProfiler/CellProfiler.git

And then
git checkout 2.2.0

git checkout master

for CP2 or 3, respectively.

pip install --user --editable .
in CellProfiler directory.


Hi Maciej!

Do you get problems when installing them both simultaneously?

I did not check recently if the prerequisites diverged between CP2 and CP3, and that could cause problems. I did not have problems in the past with a developers version of CP3, but maybe things changed now. Otherwise they should just work alongside each other.

All the best, Mario


I’m not a Ubuntu expert, but could you use something like Pyenv or Conda to create multiple environments and then just activate the one you need?


Yes, this is how you could solve the prerequisites issue! Or if you want to be a bit drastic, you can use docker to get cleanly separated environments (for example to try different python versions and stuff).


As you can install cellprofiler from source without much trouble in your current setup, following @bcimini’s advice of creating python virtual environments, I think the easiest way to generate two independent environments is to use a lightweight tool like tox. tox also gives you the flexibility of messing around with the source and seeing the immediate changes:

Create a file called tox.ini in the git root directory to tell tox how to create two environments “cp2” and “cp3”:

# tox.ini

whitelist_externals = bash

# On my ubuntu machine, it looks like cellh5 neglects to install its pandas dependency.
deps =
commands = 
    bash die_if_ref_not 2.2.0
    {envpython} setup.py {posargs:install}

usedevelop = True
commands =
    bash die_if_ref_not v3.0.0
    {envpython} setup.py {posargs:install}

In the git root directory, create the die_if_ref_not shell script to sanity check our git branch:

# die_if_ref_not

if ! [[ "$(git describe --tags --abbrev=0)" == "$1" ]]
    echo >&2 "Error: 'git checkout' the appropriate tag or branch for $1 first!"
    exit 1

Then install tox:

pip install --user tox

And run tox to generate the environments:

# Add tox to our path, if necessary
git checkout 2.2.0
tox -e cp2
git checkout master
tox -e cp3

You should see a bunch of messages about packages being installed and you can find your cellprofiler execultables in the newly created hidden .tox directory at:

.tox/cp2/bin/cellprofiler --version
.tox/cp3/bin/cellprofiler --version

Enjoy :slight_smile:


Ah, with tox trying to launch the graphical interface, I see it complains about the missing wx python dependency, and wxPython isn’t easy to compile inside a virtualenv; it doesn’t yet have a generic GNU/Linux wheel available on pip and the compile (at least on Ubuntu 16.04) is broken.

A super hacky workaround is using the system wx in the virtualenv:

for env in cp{2,3}; do
    for target in wx-3.0-gtk2 wx.pth; do
        ln -s {/usr/lib/python2.7/dist-packages/,.tox/$env/lib/python2.7/site-packages/}$target

Now the GUIs for cellprofiler 2 and 3 both launch :slight_smile:

For good measure, we should modify our tox.ini to do the symlink step otherwise if run tox again it would clear the directories deleting the symlinks. Automate all the things!

# tox.ini

whitelist_externals = bash

# On my ubuntu machine, it looks like cellh5 neglects to install its pandas dependency.
deps =
commands = 
    bash die_if_ref_not 2.2.0
    {envpython} setup.py {posargs:install}
    bash link_wx {envname}

usedevelop = True
commands =
    bash die_if_ref_not v3.0.0
    {envpython} setup.py {posargs:install}
    bash link_wx {envname}
# link_wx

for target in wx-3.0-gtk2 wx.pth; do
    if ! [[ -e $venv/$target ]]; then
        ln -s $sys/$target $venv/$target

I also tried the alternative of using anaconda, but ran into an issue of undefined symbol: PyFPE_jbuf:

conda create -n cp2 wxPython  h5py matplotlib numpy pytest pyzmq scipy scikit-{image,learn} pandas pillow six future
source activate cp2
pip install git+http://github.com/cellprofiler/cellprofiler@2.2.0

At this point running cellprofiler gives the error:

(cp2) $ cellprofiler
Traceback (most recent call last):
  File "/home/omsai/miniconda2/envs/cp2/bin/cellprofiler", line 11, in <module>
    load_entry_point('CellProfiler==2.2.0', 'console_scripts', 'cellprofiler')()
  File "/home/omsai/miniconda2/envs/cp2/lib/python2.7/site-packages/cellprofiler/__main__.py", line 175, in main
    from cellprofiler.utilities.cpjvm import cp_start_vm
  File "/home/omsai/miniconda2/envs/cp2/lib/python2.7/site-packages/cellprofiler/utilities/cpjvm.py", line 3, in <module>
    import javabridge
  File "/home/omsai/.local/lib/python2.7/site-packages/javabridge/__init__.py", line 38, in <module>
    from .jutil import start_vm, kill_vm, vm, activate_awt, deactivate_awt
  File "/home/omsai/.local/lib/python2.7/site-packages/javabridge/jutil.py", line 157, in <module>
    import javabridge._javabridge as _javabridge
ImportError: /home/omsai/.local/lib/python2.7/site-packages/javabridge/_javabridge.so: undefined symbol: PyFPE_jbuf

The best explanation for the cause of the error seems to be compilation with --with-fpectl:

For javabridge maybe that relates to how jars/cpython.jar was created? I don’t know.

In any case, using simple virtual environments created by tox works.


Incredible! It works. Thank you so much for this detailed comment.

Here’s some Java weirdness… When I keep

export JAVA_TOOL_OPTIONS=-Xss1280k

in .bashrc, I encounter following problems:

  • for CP2, after executing tox -e cp2, I get the error “UnboundLocalError: local variable ‘jdk_dir’ referenced before assignment”,

  • for CP3, tox completes the install but then I get the same error as above when I run cellprofiler.

When I leave the export... line out, everything works fine. It’s strange because I had JAVA_TOOL_OPTIONS set all the time and both CP2 and 3 worked fine when installed in the standard way (non-tox).


I also see the same error if I set JAVA_TOOL_OPTIONS or _JAVA_OPTIONS. I’m not sure why the virtual environment should matter. I suspect if you drop into a debugger using pdb you can get more information. For example:

# In the git root directory
.tox/cp3/bin/pip install ipython
# Now that we have a debugging tool, reproduce the error:
JAVA_TOOL_OPTIONS="-Xss1280k" .tox/cp3/bin/ipython
import cellprofiler
# Make sure we are using our development version:
# Out: 'cellprofiler/__init__.pyc'
from cellprofiler.__main__ import main
# Error!
# Now we are at the stack frame which has the error.
# Play around to see what's wrong.
# Ah!  So The problem is javabridge always wants "java" or "openjdk"
# as the first line, but having our JAVA_*OPTIONS makes the first line:
# "Picked up JAVA_TOOL_OPTIONS: -Xss1280k"

If you would like to do the honors, you can file a bug upstream with the javabridge devs:


Honestly, I wouldn’t know how to describe this error, so go ahead.

Still, there’s one problem. CP3 is running fine through tox, but in CP2 I come across the following error. When viewing of module results is on (the “eye” icon is open), I only get blank pop-up window without images. Below is the error when I try to see results of IdentifyPrimaryObjects:

Exception in CellProfiler core processing
Traceback (most recent call last):
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/backends/backend_wx.py", line 944, in _onPaint self.draw(drawDC=drawDC)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/backends/backend_wxagg.py", line 48, in draw FigureCanvasAgg.draw(self)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 430, in draw self.figure.draw(self.renderer)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/artist.py", line 55, in draw_wrapper return draw(artist, renderer, *args, **kwargs)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/figure.py", line 1299, in draw renderer, self, artists, self.suppressComposite)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/image.py", line 138, in _draw_list_compositing_images a.draw(renderer)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/artist.py", line 55, in draw_wrapper return draw(artist, renderer, *args, **kwargs)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/axes/_base.py", line 2437, in draw mimage._draw_list_compositing_images(renderer, self, artists)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/matplotlib/image.py", line 138, in _draw_list_compositing_images a.draw(renderer)
File "/home.nis/NIScp/CellProfiler/.tox/cp2/local/lib/python2.7/site-packages/cellprofiler/gui/cpfigure.py", line 2011, in draw im = matplotlib.image.fromarray(image, 0)
AttributeError: 'module' object has no attribute 'fromarray'


What we loosely do is to follow the base Ubuntu package installation of:

Then we create a virtualenv for cp22 and cp3 using --system-site-packages, for example:

mkdir ~/v/
virtualenv --system-site-packages ~/v/cp22
virtualenv --system-site-packages ~/v/cp3

Depending on whether or not you want to actually work with the source code you can either have multiple checkouts, a bare checkout, or use one of the aforementioned pip install styles to install a commit directly. You would activate the appropriate virtualenv for the installation beforehand of course and then deactivate as necessary.

Keep in mind that with the stable branch’s dependency specifications and the progression of development on 3.x there are a few snakes lurking in the grass. We have attempted to address a couple of the most significant instances here: