MySQL ExportToDatabase error when running multiple jobs on cluster

exporttodatabase

#1

Hi CellProfiler Team,

Could you please provide us some recommendations as to how we should modify our Mysql database to handle running CellProfiler 2.2.0 across multiple jobs on our cluster? We are running the software simultaneously across 90 jobs and are experiencing the following error:

Thu Jun 7 11:08:27 2018: Image # 1475, module CreateBatchFiles # 28: 0.00 sec
Failed to write measurements to database
Traceback (most recent call last):
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py”, line 2950, in write_data_to_db
execute(self.cursor, stmt, return_result=False)
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py”, line 255, in execute
cursor.execute(query)
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/MySQLdb/cursors.py”, line 205, in execute
self.errorhandler(self, exc, value)
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/MySQLdb/connections.py”, line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (1205, ‘Lock wait timeout exceeded; try restarting transaction’)
Failed during post-group processing for module ExportToDatabase
Traceback (most recent call last):
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/cellprofiler/pipeline.py”, line 2359, in post_group
module.post_group(workspace, grouping)
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py”, line 1977, in post_group
image_number = image_number)
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py”, line 2950, in write_data_to_db
execute(self.cursor, stmt, return_result=False)
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py”, line 255, in execute
cursor.execute(query)
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/MySQLdb/cursors.py”, line 205, in execute
self.errorhandler(self, exc, value)
File “/config/binaries/miniconda3/4.4.10/envs/cellprofiler_v2.3.1/lib/python2.7/site-packages/MySQLdb/connections.py”, line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (1205, ‘Lock wait timeout exceeded; try restarting transaction’)
Failed to stop Ilastik
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored

Once a few of the jobs have died the remaining ones complete successfully. We don’t run into any errors when running the same pipeline on the cluster, but using ExportToSpreadsheet or SaveImages.

Many thanks,

Karla