Crash in Cell Profiler Linux on cluster


#1

Got this error (CP would just crash, no warning, when switching from a terminal window back into the GUI) several times in a row running CP on the cluster. I’m using the new build: Linux v1.0.4532.


   Segmentation violation detected at Thu Dec 28 15:50:18 2006

Configuration:
MATLAB Version: 7.1.0.183 (R14) Service Pack 3
MATLAB License: unknown
Operating System: Linux 2.6.9-22.7hp.1sp.XCsmp #1 SMP Thu Mar 23 21:35:55 EST
2006 x86_64
Window System: Hummingbird Communications Ltd. (7100), display localhost:11
.0
Current Visual: 0x23 (class 4, depth 24)
Processor ID: x86 Family 15 Model 5 Stepping 8, AuthenticAMD
Virtual Machine: Java 1.5.0 with Sun Microsystems Inc. Java HotSpot™ Clien
t VM mixed mode
Default Charset: UTF-8

Register State:
eax = 00000000 ebx = e5bd461c
ecx = 081c9e50 edx = f6dd707c
esi = ecbaf8f0 edi = 08172160
ebp = f4815ddc esp = f4815dd4
eip = e5bca505 flg = 00010292

Stack Trace:
[0] libmwjmi_mi.so:0xe5bca505(0x0817221c, 0xf4815e10, 0xf4815dec, 0)
[1] 0xeeae643b(0xecbb0480, 0xeeae4589, 0xf4815e18, 0xecbaf96b)
[2] 0xeeae09fa(0xe6a75700, 0xe6a75700, 0xe82579c8, 0xf4815e48)
[3] 0xeeae09fa(0, 0, 0, 1)
[4] 0xeeade157(0xf4815ef0, 0xf48160c4, 10, 0xecbafd78)
[5] libjvm.so:0xf37628ec(0xf48160c0, 0xf4815f94, 0xf4815fe0, 0x08172160)
[6] libjvm.so:0xf3851378(0xf3762730, 0xf48160c0, 0xf4815f94, 0xf4815fe0)
[7] libjvm.so:0xf376271f(0xf48160c0, 0x08172714, 0xf4815fe0, 0x08172160)
[8] libjvm.so:0xf3784122(0x0817221c, 0xf48160c0, 0, 0)
[9] libjvm.so:0xf37791a2(0x0817221c, 0x08f851a4, 0x0906f334, 0xf4816138)
[10] libmwjmi.so:mljCallStaticVoidMethod(_jclass*, _jmethodID*, …)~(0x08f851
a4, 0x0906f334, 0, 0) + 62 bytes
[11] libmwjmi.so:0xf673c7cf(0xffffffff, 0xf48161e0, 96, 0xf5fc0a24)
[12] libmwuix.so:0xf5ed47ab(0xf70e2350, 0, 0x01655e05, 0xf568774c)
[13] libmwuix.so:0xf5ed1bf8(0, 15, 0, 0)
[14] libmwuix.so:0xf5ecfcbf(0, 0x080b21e8, 15, 0)
[15] libmwuix.so:0xf5ecae98(0x080b21e8, 15, 0x00693441, 0x0072059c)
[16] libmwuix.so:0xf5ed50c6(0, 0xffffffff, 0, 0xf7de09b0)
[17] libmwuix.so:ws_ProcessPendingEvents(int, int)~(0, 0xffffffff, 0xf4816410,
0xf6dd9110) + 147 bytes
[18] libmwservices.so:svWS_ProcessPendingEvents~(0, 0xffffffff, 0, 0x08092de4
"/blast/bwittner/cellp.2/MCR/v73/…") + 39 bytes
[19] libmwmcr.so:0xf6dd273c(0xffffd070, 0, 0, 0)
[20] libpthread.so.0:0xf7ec8341(0xf4816bb0, 0, 0, 0)

Please follow these steps in reporting this problem to The MathWorks so
that we have the best chance of correcting it:

  1. Send this crash report to segv@mathworks.com for automated analysis.
    For your convenience, this information has been recorded in:
    /shr/home/afinley/matlab_crash_dump.1661

  2. Also, if the problem is reproducible, send the crash report to
    support@mathworks.com along with:

    • A specific list of steps that will reproduce the problem
    • Any M, MEX, MDL or other files required to reproduce the problem
    • Any error messages displayed to the command window
      A technical support engineer will contact you with further information.

Thank you for your assistance. Please save your workspace and restart
MATLAB before continuing your work.


#2

Hi,
Unfortunately, we receive the same error. To get around it (a temporary fix), you can restart the job.

It is unclear where the bug is coming from, but we are working to troubleshoot the issue. Just to be sure, are you setting the CreateBatchFiles to use Matlab or the compiled version of CPCluster?

~martha


#3

Hello All,

I am not running CellProfiler on a cluster (yet), but simply the Linux compiled version. I also get this exact same error using version 1.03544 and version 1.04628 of CellProfiler. Has this error been corrected in later versions or is a fix known?

Thanks much,
Ulric


#4

Hi all,
We believe this error has been fixed. Please try the newest version of CPCluster, which can be downloaded off of the website.

Martha


#5

Sorry Martha,

I believe I was a little unclear. I am running Linux on a single machine, so I am not using CPCluster. I am just using CellProfiler compiled for Linux and I had the same problem as listed by AFinley. CellProfiler crashes and gives the fault seemingly at random with the same error message.

Again, this is just for regular CellProfiler, not for CPCluster.

Thanks for all the help,
Ulric


#6

Hello,
Are you receiving this error with a specific module, or any module in CellProfiler?
thanks,
martha


#7

No,

It is truly random when this bug occurs. As a test, I simply launched CellProfiler and left it active for a few minutes (not analyzing anything, not doing anything, no loading… really, just launched) and it crashed. I have been able to analyze a few images and have it work and continue operating for a few minutes, then it crashes. I have had it crash in the middle of analyzing a set of images.

There really doesn’t seem to be any trigger.


#8

Hi,
We are looking into this and will get back to you when we have a better idea of what is going on.

thanks for your patience.
~martha


#9

I have tested our latest release and I cannot reproduce this error. Which version were you using when the crash happened?


#10

Hello Martha,

I am using Ubuntu 6.10 and I have used both version 1.03544 and version 1.04628 of CellProfiler. I am also using a Mac Pro (early 2008) and using VMware to virtualize the machine. I am hoping to create a cluster at home.

Having said that, I was trying another way of setting up a cluster on the Mac Pro by using the MacIntel version of CellProfiler and Xgrid. However, I ran into the following problem when I added the “CreateBatchFiles” module:



         Assertion detected at Thu May  1 22:37:19 2008

Assertion failed: charCount == nCodeUnits, at line 2616 of file “marshal.cpp”.

Configuration:
MATLAB Version: 7.4.0.34341 (R2007a)
MATLAB License: unknown
Operating System: Darwin 9.2.2 Darwin Kernel Version 9.2.2: Tue Mar 4 21:17:34 PST 2008; root:xnu-1228.4.31~1/RELEASE_I386 i386
Window System: The X.Org Foundation (10300000), display :0.0
Current Visual: 0x23 (class 4, depth 24)
Virtual Machine: Java 1.5.0_13 with Apple Inc. Java HotSpot™ Client VM mixed mode, sharing
Default Charset: UTF-8

Stack Trace:
[0] libmwmcr.dylib:mnGetLogoutScriptName()~(0x0078f640 “Assertion failed: charCount == n…”, 0xb7ff82b3 “Assertion failed%s at line %d of…”, 0x000c3344 “: charCount == nCodeUnits,”, 2616) + 342 bytes
[1] libmwmcr.dylib:mnGetLogoutScriptName()~(0x000c3344 “: charCount == nCodeUnits,”, 0x000c2f5c “marshal.cpp”, 2616, 0) + 652 bytes
[2] libut.dylib:ut_assert~(0x000c3344 “: charCount == nCodeUnits,”, 0x000c2f5c “marshal.cpp”, 2616, 0x00cc8bc0) + 70 bytes

Please follow these steps in reporting this problem to The MathWorks so
that we have the best chance of correcting it:

  1. Send this crash report to segv@mathworks.com for automated analysis.
    For your convenience, this information has been recorded in:
    /Users/ulrickwan/matlab_crash_dump.32414

  2. If the problem is reproducible, please submit a bug report via:
    mathworks.com/support/contac … est_1.html

Thank you for your assistance. Please save your workspace and restart
MATLAB before continuing your work.

Error in ==> CreateBatchFiles at 274

Error in ==> CellProfiler>AnalyzeImagesButton_Callback at 9409

Error in ==> CellProfiler>gui_mainfcn at 11138

Error in ==> CellProfiler at 55

??? Error using ==> CellProfiler(‘AnalyzeImagesButton_Callback’,gcbo,],guidata(gcbo))
Index exceeds matrix dimensions.

??? Error while evaluating uicontrol Callback

I have run a case with 384 images with the same pipeline but without the “CreateBatchFiles” module and there were no errors with the analysis or the output. I have tested this on my Macbook with 2GB of RAM and on my Mac Pro with 16GB of RAM. In both cases, the analysis is fine without “CreateBatchFiles” and the analysis fails with “CreateBatchFiles”. I am using the latest CellProfiler version 1.0.5122 for Intel Macs. I am also using Leopard.

Sorry for putting you guys through all of this. I am really just trying to get a cluster going over here.

Thanks so much for all of the assistance so far,
Ulric


#11

Sorry for the delay… I’m not sure if this was a problem on the older versions of CellProfiler (versus the newer version, 1.0.5122). Second, we are working with our IT group to have a workstation upgraded to Leopard. Hopefully, we can start releasing CellPRofiler for Leopard in the near future. It is possible that the error that you are seeing is related to your OS.

~martha