Sampling rate guidelines (VAD, ASR)

[pehonnet]

Hello,

Thanks again for this great toolkit. I have had good success with "standard" 16kHz based model, even trained on data which is actually 8kHz upsampled, but I am not able to get anywhere when trying to train a model directly on 8kHz data.

I tried to look a bit in the repo but there does not seem to be any examples using 8kHz data (only a tutorial on telephony ASR, which just overwrites the model sampling rate, no changes in feature extraction parameters). Of course I could use the 16kHz model, but this means I would needlessly resample data which could actually stay in 8kHz. My current work is on the VAD using marblenet, and I have tried a few different settings for the feature extraction part but cannot get results as good as with the 16kHz default parameters.

Do you have any recommendations to use directly 8kHz data? Or any idea why an upsampled 16kHz signal would provide anything more than the 8kHz one?

Thanks!

[titu1994]

We've seen something similar, hence the telephony tutorial recommending to always use 16khz or more.

We have trained pure 8khz models but their wer is always worse than 16khz, no exceptions.

As to why, maybe it's spec Aug being overly strong for 8k and therefore needs tuning - but that is just by personal conjecture. There's no theory suggesting proper reason.

[pehonnet]

Thanks, ok that's interesting. It looks like many other frameworks also stick to 16kHz but without any proper justification. Maybe I will try to compare 8/16kHz without specaugment (although performance might be worse of course)

[pehonnet]

Just a follow up in case of interest: as you suggested @titu1994 I have investigated the use or not of specaugment for training, and what I found out is:

• without specaugment, 8kHz and 16kHz model perform a bit different (8kHz model has better precision, 16kHz model has better recall for the same inference parameters, I didn't try to tune them): 8kHz model: PRC=0.74; RCL=0.87 / 16kHz model: PRC=0.66; RCL=0.96
• with specaugment, the 16khz model performs better than without it (increased precision), but the 8kHz performs quite poorly: 8kHz model: PRC=0.37; RCL=0.33 / 16kHz model: PRC=0.79; RCL=0.96

So it seems your intuition makes sense and possibly tuning specaugment parameters for 8kHz could help. Do you have any hint on what way to go? Like maybe decreasing these values?

   freq_width: 15
    time_width: 25
    rect_masks: 5
    rect_time: 25
    rect_freq: 15