production inference server with continuous batching?

[devidw]

Hey guys,

Does anybody know a production-ready TTS inference server that works with StyleTTS2 or could be modified to work with StyleTTS2 model inference.

Looking for something to use in a near real-time app, therefore with high throughput and being capable of handling requests in parallel, using continuous batching or other queuing mechanisms to batch input into the model for fastest response times.