Mitigate "Augmix" from mmclassification to mmpretrain

[EdyQiu]

Hello, I encountered an issue while trying to migrate the "Augmix" class from the Misclassification to the MMpretrain. "Augmix" is an augmentation method that I wrote myself and placed in the PIPELINE module.

The implementation is straightforward, but when loading 1 image, 3 augmented images will be the output. it returns a list instead of a dictionary, which causes issues in the data-loading pipeline.

To resolve this issue, I modified the "Compose" class in the data-loading pipeline to accept a list as input.

However, when attempting to migrate the code from Misclassification to MMpretrain, I encountered several problems.

The most significant issue is that I could not find an equivalent function to "Compose" in mmpretrain. As a result, I am unable to resolve the problem of "Augmix" returning a list instead of a dictionary, causing 'PackInputs' to fail to work.

So I am here to ask for some help, is there an equivalent function to "Compose" in mmpretrain. Or is there any clever idea to implement "Augmix" in MMpretrain.

Thanks a lot for the help!

[EdyQiu]

Sorry I used the wrong word "Mitigate", it should be "Migrate"

[EdyQiu]

Another problem: image normalization is conducted in Augmix, soit has conflict with "data_preprocessor", but I guess I can simply not use the data_preprocessor, and use "normalize" in data pipeline if needed.

[Ezra-Yu]

I'm very sorry for the inconvenience caused by this upgrade. Compse is in mmcv. Specifically, you can access https://github.com/open-mmlab/mmcv/blob/89a264527e3dc9c5eebed6195faa709d446c7a9c/mmcv/transforms/wrappers.py#L40

to disuse normalize in data_preprocessor:

data_preprocessor = dict(
    # RGB format normalization parameters
    mean=None,
    std=None,
    # convert image from BGR to RGB
    to_rgb=True,
)

I notice that Augmix has been supported in torchvision refer to: https://pytorch.org/vision/stable/generated/torchvision.transforms.AugMix.html#torchvision.transforms.AugMix

Maybe you can use that, refer to PR #1265

[EdyQiu]

Thank you for your previous advice! I have decided to implement Augmix myself to facilitate further modifications. However, I am still struggling with the issue of multiple image outputs during the training pipeline. When loading one image, the actual batch has three additional images.

I attempted to set a breakpoint in "Compose" in mmcv, but it did not stop there. I am now unsure what to try next.

I apologize for bothering you again, but could you please offer any suggestions or guidance?

[Ezra-Yu]

why not just return a mixed image in Augmix and put some params in result['augmix_param'].

I can not get why you want to return the list of images.

[EdyQiu]

While since Augmix is a module in the loading PIPELINE, it should return result, which is a dictionary with keys(['img_path', 'gt_label', 'sample_idx', 'img', 'img_shape', 'ori_shape', 'flip', 'flip_direction']), which contains one image only

but Augmix should return 3 images: the original copy, the first augmentation, the second augmentation(they are used to compute consistency loss). So I am confused about how to return 3 images.
In mmclassification, I return a list with 3 "results" to return all 3 images, and modified "Compose", so that 3 elements in "return" will come to the next module (e.g. "ImageToTensor") one by one

Would you mind explaining further about "return a mixed image in Augmix and put some params in result['augmix_param']." ?

[EdyQiu]

I solved the problem by modifying the following files in mmpretrain, to replace the equivalent parts in "Compose" in mmclassification:

Thanks for your help !