[BUG] Unable to load annotation in ITK snap

[PolarBean]

Describe the bug
When downloading an atlas, I sometimes wish to view the reference and annotation in ITKsnap. This is likely the most popular software for interacting with atlases. The reference loads fine as a generic ITK image, however the annotation will not load.

To Reproduce
Download any atlas from brainglobe. I have tried with multiple, most recently the 50um Allen. Load the reference and select 'Generic ITK Image' as the format. Do the same for the annotation. The annotation will not load.

I'm not sure what the solution here is. I think we should be able to find some combination of settings to import the annotations as a raw binary. It may be that the header of the annotations is wrong. I will look into this.

[adamltyson]

Does this happen with the other, non-Allen atlases? The Allen atlases have some very high values that the others don't.

[PolarBean]

True I have tried only with different allen atlases (including the just released allen based barrel). The high values in the allen can usually be handled by using unsigned 32 bit integers.
I tried to insert the correct values manually and am getting quite strange results. I will try now with non allen

[PolarBean]

Still allen based but for the Barrel atlas the segmentation still appears misaligned once loaded in itk snap. also while the allen required Big Endian this one requires little endian.

It seems like there is differing metadata between annotations. Is header information stored when creating bg annotations?

[adamltyson]

The barrel atlas isn't finished yet (it should have been released at version 0).

I don't think we specifically save any information in the tiff headers.

I don't use itk-snap, do you know why this software has an issue, but others don't?

[PolarBean]

Not really sure. I looked into it a bit more but didn't solve it. I think itksnap just works better with NRRD and NII.

[adamltyson]

Is it possible that the issue is on the itk-snap side? I.e. it isn't loading tiffs correctly?

We can certainly make changes on our side (if we know what those are!) if needed, we're re-packaging lots of atlases anyway. We could also move to another format if needed for v2 of the API. We chose tiff because it's the format that target users are most familiar with.

[PolarBean]

pyushkevich/itksnap#137 I opened an issue about whether its a problem with itksnap.

I am not sure about changing the format. That would of course be a pretty breaking change. I think Nifti has some advantages over tiff as it is a format designed for neuroimaging data rather than general imaging. other neuroscience standardization initiatives such as BIDS for example say all volumes must be stored as Nifti. But I understand if others disagree and prefer tiff.

[adamltyson]

Thanks for raising that.

Fields like MRI use Nifti, but BrainGlobe (currently, and mostly) caters to histology analysis, which mostly uses tiff or other microscopy formats. Nifti is a very specific format that works well within it's own ecosystem, but it's confusing to those from outside the field. Some tools built for Nifti also break down if you try to use them with some of the whole-brain datasets (200GB+) that are commonly used with BrainGlobe.

I am not sure about changing the format. That would of course be a pretty breaking change.

In theory, it wouldn't be. If people are accessing the data via the API (and not directly) then we could do this without the users even noticing!

In the future, it would be good to move to something like OME-Zarr, but that may not help with the itk-snap issue.

[PolarBean]

In theory, it wouldn't be. If people are accessing the data via the API (and not directly) then we could do this without the users even noticing!

That's pretty great. I suppose the question is whether brainglobe files should ever be directly accessed by the user. Ie; downloading via brainglobe and then opening the files independently using some other program. If not then this is a non issue since the api will return an array correctly regardless of filetype. It would be useful to have a repository of atlases and templates which is very close to what brainglobe is but perhaps out of scope.
If there is scope for brainglobe to serve as a respository then I think the main benefits of Nifti over tiff are compatibility with almost all volumetric imaging software (all that ive seen). Further even though it may have been designed for MRI a lot of the atlases were created in Nifti compatible software such as ITK snap (The allen and the waxholm). So atlases distributed via brainglobe as tiff would no longer be compatible with the software they were created in. Further the header also includes useful information that tiff does not such as datatype and affine orientation. But again these points are moot if acting as an atlas repository is out of scope.

[adamltyson]

I suppose the question is whether brainglobe files should ever be directly accessed by the user.

People can, of course, but it's not officially supported. By default, atlases are saved into a hidden directory.

If there is scope for brainglobe to serve as a respository then I think the main benefits of Nifti over tiff are compatibility with almost all volumetric imaging software (all that ive seen)

I'd be interested to know if this is true across domains. It's certainly true for tools built for medical imaging, but not sure about other fields. For example, tiffs are supported "out of the box" in napari, but Nifti files require a specific plugin.

So atlases distributed via brainglobe as tiff would no longer be compatible with the software they were created in

Ideally atlases could "go back" into itk-snap etc. However, I wouldn't want to be in the situation where the atlases and experimental data require different tools to load. We focus heavily on microscopy data because:

• That's where we think the need is
• The MRI community already has lots of excellent tools

We didn't envision BrainGlobe as a repository initially, but as we expand, it may be a good idea. No reason why we couldn't store multiple formats though, or convert on the fly.

[PolarBean]

Fair points, I think converting on the fly is a good compromise!