Has anyone tried deploying whisper model on the NVIDIA Triton Inference Server?

[Kianit-bit]

Can I run whisper on Triton?
Any link to example code will be very helpful.

[mhbassel]

In case you're interested, there's a related discussion in OpenAI github: openai/whisper#1505
But still no one shared any code examples.

[saitejamalyala]

#7847

Above Has a example using python backend (there are better ways to do it as well). But only http endpoint works unfortunately.

[dineshveguru]

Hi @saitejamalyala, Could you please tell me how to use the API. I hosted it and running fine. But, getting errors while using it

[saitejamalyala]

@dineshveguru , Can you share a gist of how you are trying to use it ?

[dineshveguru]

My intended use case is to send audio file url to the API, so that I can download and store the audio file locally in the server and then I can process it and then send the transcription through the API.
This is how I tried to use it.

{
    "INPUT": "http://audiofile.wav"
}

Thank you.

[saitejamalyala]

hf pipeline has the capability to use the url as the source, do you have more information regarding the error ?