Exception in normalization step


#1

Hi,
I am trying to follow the steps described in the SI of the NatureProtocols paper to get per-Well features with a new data set. The data is in a MySQL database.


This step seems to run fine:
python2 -m cpa.profiling.cache -r ../input/cc_bin2_correct.properties ../input/cache ""
WARNING:root:PROPERTIES WARNING (channels_per_image): No value(s) specified. CPA will assume 1 channel per image. INFO:root:PROPERTIES: Using default image_buffer_size=1 INFO:root:PROPERTIES: Using default tile_buffer_size=1 WARNING:root:PROPERTIES WARNING (well_format): Field was not defined, using default format of "A01". INFO:root:[MainThread] Connecting to the database... Per_Object_checked Per_Object INFO:root:Ignoring columns: ['ImageNumber', 'ObjectNumber'] Features: 100% |###############################################################################| Time: 0:13:00 Not performing normalization because not predicate was specified.


But the next step gives an error:
python -m cpa.profiling.normalization ../input/cc_bin2_correct.properties ../input/cache ""
WARNING:root:PROPERTIES WARNING (channels_per_image): No value(s) specified. CPA will assume 1 channel per image. INFO:root:PROPERTIES: Using default image_buffer_size=1 INFO:root:PROPERTIES: Using default tile_buffer_size=1 WARNING:root:PROPERTIES WARNING (well_format): Field was not defined, using default format of "A01". INFO:root:[MainThread] Connecting to the database... Per_Object_checked Per_Object Traceback (most recent call last): File "/home/pahl/anaconda3/envs/cellprof/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/home/pahl/anaconda3/envs/cellprof/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/home/pahl/dev/github/CellProfiler-Analyst/cpa/profiling/normalization.py", line 266, in <module> normalizer._create_cache(predicate) File "/home/pahl/dev/github/CellProfiler-Analyst/cpa/profiling/normalization.py", line 81, in _create_cache self._create_cache_params(predicate, resume) File "/home/pahl/dev/github/CellProfiler-Analyst/cpa/profiling/normalization.py", line 141, in _create_cache_params controls = self._get_controls(predicate) File "/home/pahl/dev/github/CellProfiler-Analyst/cpa/profiling/normalization.py", line 90, in _get_controls cpa.properties.image_table, predicate)): File "/home/pahl/dev/python2/pkg_links/cpa/dbconnect.py", line 64, in fn return f(db, *args, **kwargs) File "/home/pahl/dev/python2/pkg_links/cpa/dbconnect.py", line 529, in execute '\nSecond exception was: %s'%(connID, query, e, e2)) cpa.dbconnect.DBException: ERROR: Database query failed for connection "MainThread" and failed to reconnect Query was: "select distinct Image_Metadata_Plate, ImageNumber from Per_Image where " First exception was: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1") Second exception was: ERROR: Database query failed for connection "MainThread" Query was: "select distinct Image_Metadata_Plate, ImageNumber from Per_Image where " Exception was: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1")


Do you have any ideas what could be the issue here?
Many thanks in advance.

Kind regards,
Axel
cc_bin2_correct.properties (6.5 KB)


#2

Can you try
python -m cpa.profiling.normalization ../input/cc_bin2_correct.properties ../input/cache "Image_Metadata_ASSAY_WELL_ROLE = 'Untreated'"

and let me know if that works?


#3

Hi @shsingh, thanks for your answer.
Unfortunately, that did not help.

Now I get these errors:
First exception was: (1054, "Unknown column 'Image_Metadata_ASSAY_WELL_ROLE' in 'where clause'") Second exception was: ERROR: Database query failed for connection "MainThread" Query was: "select distinct Image_Metadata_Plate, ImageNumber from Per_Image where Image_Metadata_ASSAY_WELL_ROLE = 'Untreated'" Exception was: (1054, "Unknown column 'Image_Metadata_ASSAY_WELL_ROLE' in 'where clause'")

Kind regards,
Axel


#4

What are the entries for image_table and object_table in your seedseq.properties files?

Also did this step complete successfully?
mysql -u [user] -p[password] [database_name] < seedseq_per_cytoplasm.sql


#5

I am beginning to understand that this approach probably does not work with my data set as it currently is (my .properties file is attached in my first post).
Thanks for your help!


#6

Update the properties file:

# ==== Database Tables ====
image_table = MultipleHairpin_2013_03_07_Analysis_Per_Image
object_table = MultipleHairpin_2013_03_07_Analysis_Per_Object

Question for you:
Did this step complete successfully?

mysql -u [user] -p[password] [database_name] < seedseq_per_object.sql