Getting OOM (out of memory) when using big files

[shamamayair]

When using big files we getting out of memory did some debug and found it happen here:

faster-whisper/faster_whisper/feature_extractor.py

Line 189 in 1b24f28

output = np.fft.rfft(input_array, n=n_fft, axis=-1, norm=norm)

its only happen on big files (few hours) on small files its working fine.

this is very basic draft of the code we using (we fo course getting the oom on the transcribe:

model = WhisperModel(model_size)

vo = VadOptions(
min_speech_duration_ms=250,
speech_pad_ms=30,
min_silence_duration_ms=5000,
)

options = {
"word_timestamps": True,
"vad_filter": True,
"condition_on_previous_text": False,
"hallucination_silence_threshold": 3,
"log_prob_threshold": -0.5,
}

options["vad_parameters"] = vo

segments, info = model.transcribe(audio_file, **options)

did you encounter such an issue ? any ideas?

[Purfview]

It was fixed in the last commit: #1198

Install the latest master:
Press "Code" button then press "Downlod ZIP", then run: pip install "faster-whisper-master.zip"
Or with git: pip install git+https://github.com/SYSTRAN/faster-whisper.git

[MahmoudAshraf97]

It was fixed in the last commit: #1198

Install the latest master: Press "Code" button then press "Downlod ZIP", then run: pip install "faster-whisper-master.zip" Or with git: pip install git+https://github.com/SYSTRAN/faster-whisper.git

I guess this might be more related to the feature extraction rather than VAD, but if the vad used less memory that might help with the problem but it will still be there

[jhj0517]

Can you please reopen this? According to jhj0517/Whisper-WebUI#424 (comment), this seems to still be reproducible on Colab with the large file ( 1 hour 40 minutes).

[Purfview]

Can you please reopen this? According to jhj0517/Whisper-WebUI#424 (comment), this seems to still be reproducible on Colab with the large file ( 1 hour 40 minutes).

Maybe it was on low memory even before VAD, what RAM usage it shows without VAD?

[jhj0517]

@Purfview I just tried to reproduce it myself on my side with 2 hours of video, but the CPU RAM was reached only 5.4 GB.
So I guess the problem is not related to this issue, it's just a problem on my side. I'm sorry for giving you confusion.

This is the peak CPU RAM data I observed with 2 hours of audio (the left "시스템 RAM" means CPU RAM):

1. when using VAD together for 2 hours of audio : 5.4GB peak RAM (CPU)
   

2. when using only faster-whisper for 2 hours of audio: 3.5GB peak RAM (CPU)
   

[pablopla]

Can you please make a new release so we don't have to install from git?