C9-Audio-Based-Interaction-Recognition

Challenge

To participate and submit to this challenge, register at the EPIC-SOUNDS Audio-Based Interaction Recognition Codalab Challenge. The labelled train/val annoations, along with the recognition test set timestamps are available on the EPIC-Sounds annotations repo. The baseline models can also be found here, where the inference script src/tools/test_net.py can be used as a template to correctly format models scores for the create_submission.py and evaluate.py scripts.

This repo is a modified version of the existing Action Recognition Challenge.

NOTE: For this version of the challenge (version "0.1"), the class "background" (class_id=13) has been redacted from the test set. The argument --redact_background is supported in evaluate.py to remove background labels from your validation set evaluation.

Result data formats

We support two formats for model results.

• List format:

[
    {
        'interaction_output': Iterable of float, shape [44],
        'annotation_id': str, e.g. 'P01_101_1'
    }, ... # repeated for all segments in the val/test set.
]

• Dict format:

{
    'interaction_output': np.ndarray of float32, shape [N, 44],
    'annotation_id': np.ndarray of str, shape [N,]
}

Either of these formats can saved via torch.save with .pt or .pyth suffix or with pickle.dump with a .pkl suffix.

Note that either of these layouts can be stored in a .pkl/.pt file--the dict format doesn't necessarily have to be in a .pkl.

Evaluating model results

We provide an evaluation script to compute the metrics we report in the paper on the validation set. You will also need to clone the annotations repo.