Default multi-class bounding box selection doesn't update label on save

[eric-tramel]

Use Case: A widget for annotating multiple Bounding Boxes within a single image, each of which can belong to one of many classes.

Problem: If you draw a bounding box, but leave the dropdown on its default value, the one-hot labelling of the bounding box will be all 0 (e.g. no label assigned). If you change the assigned class in the drop down to any other class and reselect the default (first) item, then the labelling will be recorded properly.

Example Innotater Call

# setup all the other variables, e.g. image list etc before this.
annotation_labels = ["default", "a", "b"]

innotater_args = { 
            "inputs": [
                ImageInnotation(image_files, path=image_path, width=image_width), 
                TextInnotation(image_files, multiline=False)
            ],
            "targets": [
                RepeatInnotation(
                    (BoundingBoxInnotation, inno_bboxes),
                    (MultiClassInnotation, inno_labels,
                        {
                            "name": "ROI Type",
                            "classes": annotation_labels,
                            "dropdown": True
                        }
                    ),
                    max_repeats=50,
                    min_repats=1
                )
            ]
}

Innotater(**innotater_args)

[danlester]

Thank you very much for this.

I'm not too sure what the best fix is.

It might be safest to insist that targets should already be valid (i.e. there should be exactly one 'hot' label selected in the first place, and you shouldn't invoke Innotater with an invalid array. It could throw an error if you do, but maybe that's not very user-friendly... If multiple ones are set, that should also be an error.

Or if a row is all zeros, maybe a new 'empty' option should appear (and be set) in the dropdown to make it clear that no real item is selected. However, you don't really want that to be an option available from the drop-down once a real selection has been made.

Or if a row is all zeros, just set the selection to the first item (as it does now) but also explicitly write this back to the array. I think the downside to this is that if you then step through a number of images without really looking at them, you inadvertently alter the data without realizing it.

So the current workaround would be for your preliminary 'loading and setup' code to set a default selection for every row in the array, and then you can at least always be sure that the data and UI is consistent.

However, I agree that there should be a better default behavior so will leave this issue open for now and will have a think and listen for any feedback.

[eric-tramel]

Thanks for the comments @danlester , I hear your points on the trickiness of the default setting.

For my use-case, I went for the last workaround you mentioned, which is creating a dummy value in the original annotation list. However, on the user-code side, it gets a bit cumbersome to keep up with this dummy value, know what it is, when to remove it, etc. Additionally, I had to put in some safety code to introduce Warnings and trigger a failed save when detecting these "blank" default fields that I introduced.

However, the sadness returns because I also have to keep up with the ordering of the labels when re-loading an annotation session from my serialization/disk format, and without this, subsequent sessions could end up having meaningful labels as a default, so its back to square-one without maintaining the ordering, too. The other approach would be to also filter out these dummy columns from the data array on save, etc, but this also creates some user-code overhead.

So, I think that having a kind of Innotater-managed default option for the multi-class selection (e.g. presenting a blank box) would be really nice in reducing this kind of overhead.

Thanks! 🙏

[danlester]

Great - thanks for your thoughts!

To be clear, your suggestion is that sometimes it is useful to have an indeterminate/blank option that can be explicitly selected to clear any 'real' selection. I guess that's slightly deeper than my thinking above which was really just about what to do if there happen to be some indeterminate inputs.

One complication is that there may not be such a natural approach to this in the non-one-hot case - i.e. when you just have an integer representing the class number. Maybe -1 works to represent 'no choice' in that case.

On balance, I wonder if the best approach is to add in this blank option by default (since this is likely to appear in one-hot arrays which are initialized as all 0), with a parameter to turn off this blank option if the developer doesn't want it to be available (in which case maybe it can throw an error for all blanks).