Image tagging and Online model

[shahareshed]

Hey,
First thanks for your awesome plugin!
Some background: I'm trying to create a detection model for bone cells, in order to create an automated tool for cell counting. It will shorten the time period of biomedical researchers in this field.
I have 3 questions. It is my first machine learning task, so those questions are quite basics :)

1. I have a set of annotated images: an image file (PNG) and Json file containing the coordinates of each class. Do you know how can I create easily based on this data tagged image (containing overlay layer) without doing manual work?
2. Can I create an online model, a model which keeps on learning based on feedback? If the answer is yes, is there a friendly user interface for giving feedback to the model? Do I have to run the model manually once I get feedback, or can it be done automatically?
3. Can I download the model and use it as an input for model written in python?

Thanks!
Shahar

[ThorstenFalk]

Hi Shahar,

1. Easily is relative. It requires an ImageJ Macro that parses the json file generates corresponding ROIs and embeds them into the image. If the Macro-language is insufficient you have to write a plugin to employ all Java features. The IJM Tutorial might be useful. I never tried to import json to ImageJ
2. You can finetune existing models with new annotations, but there is no convenient interface for active learning as you describe it. This would require two ingredients: 1. a relevance heuristics that points you to regions where annotations are most useful and 2. a quicker workflow that on-click can run the same finetuning with the same settings only using a new ROI set. Definitely useful but out-of-scope of this plugin.
3. The model is anyways saved to disk in hdf5 format. caffe will directly read it, for tensorflow and pytorch you will have to use wrapper scripts, that create a model from the caffe prototxt format with hdf5 weights. There might be off-the-shelf converters, but I never tried one of those. Usually I read the weights manually using h5py and assign them to the corresponding layers. Not scalable, but for the narrow family of U-Nets it was the simplest solution for me.

[shahare7]

Hey,
Thanks a lot.
Regarding 1 - can you elaborate a bit more about the subject? I don't know even where to start.. what should be the input and the output of ImageJ macro? What is it ROI? a format? How do you usually tag images using ImageJ?
2 + 3 are pretty clear.
Thanks a lot for your help, I appreciate it.