Failure to load OME-TIFF sample data


#1

I posted this as a reply to a related post, but am creating a new one in case that gets buried. Apologies for the duplication!

i tried loading this OME example file and see the same problem. here is part of its XML:

<Pixels BigEndian="true" DimensionOrder="XYZCT" ID="Pixels:0" SizeC="3" SizeT="7" SizeX="439" SizeY="167" SizeZ="1" Type="int8">

i.e. 7 time points, 3 channels for each. here are my steps:

  • Images: add this tif
  • Metadata: extract metadata from image file headers
  • click Update metadata
  • click Update -> 21 rows appear (SizeT * SizeC, or in this case, the number of images in the file), but all say C = 0 – i.e. it is interpreting OME’s own sample data file incorrectly by assigning all planes to one channel)
  • NamesAndTypes -> metadata does have C matching 1 (or even 0, the only one appearing in the Metadata module)
  • click Update -> “Sorry your pipeline doesn’t produce any matching images”

i also see this (like the original poster):

    Exception in thread "Thread-0" loci.common.services.ServiceException: java.lang.
IllegalArgumentException: -1.13623475E12 must not be null or non-negative.
        at loci.formats.services.OMEXMLServiceImpl.createRoot(OMEXMLServiceImpl.java:384)
        at loci.formats.services.OMEXMLServiceImpl.createOMEXMLRoot(OMEXMLServiceImpl.java:350)
        at org.cellprofiler.imageset.ImageFile.setXMLDocument(ImageFile.java:166)
        at org.cellprofiler.imageset.ImagePlaneMetadataExtractor.extract(ImagePlaneMetadataExtractor.java:319)
Caused by: java.lang.IllegalArgumentException: -1.13623475E12 must not be null or non-negative.
        at ome.xml.model.primitives.NonNegativeFloat.<init>(NonNegativeFloat.java:48)
        at ome.xml.model.primitives.PositiveFloat.<init>(PositiveFloat.java:46)
        at ome.xml.model.primitives.PositiveFloat.valueOf(PositiveFloat.java:64)
        at ome.xml.model.Pixels.update(Pixels.java:259)
        at ome.xml.model.Pixels.<init>(Pixels.java:166)
        at ome.xml.model.Image.update(Image.java:355)
        at ome.xml.model.Image.<init>(Image.java:157)
        at ome.xml.model.OME.update(OME.java:304)
        at ome.xml.model.OME.<init>(OME.java:151)
        at ome.xml.meta.OMEXMLMetadataRoot.<init>(OMEXMLMetadataRoot.java:68)
        at loci.formats.services.OMEXMLServiceImpl.createRoot(OMEXMLServiceImpl.java:379)

#2

Hi,
I think it’s exactly the same problem I posted about recently here
Best
Carlos


#3

Hello all,

I was examining the error for a while… Sorry for the delay.
The example you show here “multi-channel-time-series.tif” actually has an issue.

Although it could be loaded “fine” in ImageJ, but when you open “Image” >> "Properties"
You can see that:
Channel © : 1
Slices (z) : 21
Frames (t): 1

Which is the exact reason why when you load into CellProfiler, you would have 21 rows and only 1 channel.
I agree that the metadata written in xml is written with 3 channels, but the properties is written incorrectly.

A quick fix in the Properties, set it to:
Channel © : 3
Slices (z) : 1
Frames (t): 7

Then save the image >> Restart CellProfiler and load the new image with correct properties.
Now you will have correct time/slice/channel when using “Extract from image file headers” as you did.


note: Python indexing starts from 0.

Back to your own data, can you try open your image with ImageJ and make sure both metadata in xml and in properties agree to same values channels/slices/time. Then save the images and open in CellProfiler.

Hope that helps you both.


File header sizeC and sizeZ always 1
#4

hi Minh - thanks a lot for the investigation.

you’re right that the properties look that way if we open the tif as a normal tif stack - without using the bioformats importer, how is FIJI supposed to know anything about the metadata?

instead, if you do Plugins -> Bio-Formats -> Bio-Formats Importer and select hyperstack you’ll see what’s expected in the file’s properties, and the viewer opens as expected with sliders for channel and time point. this is what it means to open an OME-TIFF as far as i can tell - honor the OME-XML metadata.

if i do as you suggest and modify the properties in FIJI, it does work in that I can load it in CP. however, the nasty side-effect is that i have deleted ALL of my OME-XML metadata.

so it seems like one of these is true:

  • CP doesn’t actually support loading OME-TIFF files that contain OME-XML (despite its support for bioformats)
  • we are failing to find/use some setting or plugin in CP that consumes and understands the OME-XML metadata

i am hoping for the latter!


#5

i should mention one of my goals is to be able to access this data directly from Omero - i assumed that being able to open the tif from the local file system was a prerequisite.


#7

You’re right that once saved by Fiji, the XML changes. Currently I’m working on how to save images with preservation of some important XML metadata, so that in the end it may look modified but not destroyed.

Or another not-so-elegant method is that you can always export and save your precious XML metadata as an individual file (.txt or .xml), then do all you need with the images without fear.

I’ll get back to you if I have some progress on this.


#8

OK, so we know where we are now - we need to implement a workaround (pre-process the file by inserting headers that CellProfiler understands) in order to load multidimensional stacks stored in OME-TIFF files into a CP pipeline.

If I decide to go this route, I can implement this workaround myself - thank you for your time!

To wrap up the issue, the next step from my perspective would be to make this an official feature request - to enable a true “Bio-Formats Import” in CP. I suppose i’m not alone in assuming that this was possible, since bio-formats is already being used in some capacity by CP. Would it be possible for you to submit such a feature request or is there some official channel that I should use?

thanks again,
david


#9

Chiming in here to clarify: we suspect the failure to load metadata properly here is a result of CellProfiler bypassing bioformats and using a standard TIF reader on your images. Because, yes, in general CellProfiler should be able to read this type of metadata.

@Minh is going to make a note to followup on this thread once he’s discussed with our software engineers!


#10

Hello, has there been any update on this? if there is a lack of interest/resources to investigate, could you please give me some debugging pointers?


#11

It would be cool to have some kind of follow-up on this, but I guess if no one else has this problem, it isn’t a high priority. Seems like a relatively common use case though.


#12

There’s definitely still some ongoing work getting better integration of OME metadata into CP, but it’s not a process that’s going to be solved overnight. You’re welcome to open up a GitHub issue at the repo I just linked about this.

Depending on if your files all have a similar structure though, there may be an easier workaround than pre-processing the images- you can take advantage of the fact that CP will import Metadata from a .CSV file to create a CSV that looks something like the following:

Frame  RealC RealT
0      1     1
1      2     1
2      3     1
3      1     2

etc etc. It’s definitely a hack, but it may be an easier hack that processing the image itself (assuming you haven’t already scripted or macro-ed that out.


Issue splitting channels with ColorToGray for images captured with MicroManager
Grouping per well triplicates