AudioSet fine-tuning weights

We release the weights fine-tuned on AudioSet (AS2M), which were originally pre-trained with M2D (masking ratio of 0.7).

• m2d_clap_vit_base-80x1001p16x16-240128_AS-FT_enconly.zip ~ mAP 0.485
• m2d_as_vit_base-80x1001p16x16-240213_AS-FT_enconly.zip ~ mAP 0.485
• m2d_as_vit_base-80x1001p16x16p32k-240413_AS-FT_enconly .zip - 0.47998, 32 kHz input
• m2d_vit_base-80x1001p16x16-221006-mr7_as_46ab246d.zip ~ mAP 0.479

All weights are 16 kHz input unless denoted.

M2D for Speech (M2D-S) Release

This release provides pre-trained weights for:

Masked Modeling Duo for Speech: Specializing General-Purpose Audio Representation to Speech using Denoising Distillation
https://arxiv.org/abs/2305.14079

Initial release and pre-trained weights

Initial release including three pre-trained weights:

• m2d_clap_vit_base-80x608p16x16-240128.zip (1.45GB)
• m2d_as_vit_base-80x608p16x16-240213-mr7.zip (1.46 GB)
• m2d_vit_base-80x608p16x16-221006-mr7.zip (1.44 GB)
• m2d_vit_base-80x608p16x16-221006-mr7_enconly.zip (Encoder only, 302 MB)
• m2d_vit_base-80x608p16x16-220930-mr7_enconly.zip (Encoder only, 302 MB)
• m2d_as_vit_base-80x608p16x16p32k-240413_enconly.zip (Encoder only, 302.17 MB, 32 kHz input)
• m2d_vit_base-80x608p16x16-221006-mr6.zip (1.44 GB)
• m2d_vit_base-80x200p16x4-230529.zip (1.45 GB)
• msm_mae_vit_base-80x608p16x16-220924-mr75.zip (976 MB)
• Example logs.

All weights are 16 kHz input unless denoted.