Midi output stream (not only file)

[mickaelistria]

I read in the doc
"""
Very few DAWs support audio input/MIDI output plugins as far as I know. This is partially why NeuralNote is an Audio FX plugin (audio-to-audio) and that MIDI is exported via drag and drop.
"""
I use a DAW that seems to allow it (Carla), and I also tried Widi VST which does allow to create a new midi source that Carla detect (as a separate input, detached from the plugin, but still present and still allowing to directly map the midi transcription).
If this pattern of creating a new virtual midi input for the plugin output can work, I would recommend to implement it.

[mickaelistria]

In my case, I don't really mind a big delay, but I use streamed audio and don't have the file before hand, I just want to be able to enable transcription "on-the-fly"

[DamRsn]

Hi,

I understand your point but I don't think that it is worth the effort, and I'm not even sure it is feasible. The inability for most DAWs to have audio in and midi out is only one of the reasons why we can't stream the midi out, even with some large delay.

As indicated in the readme, we would need to re-implement completely the CQT (currently computed using ONNXRuntime) so that it can work in streaming. And we would also need to change the algorithm used to create note events from Basic Pitch's outputs.

Also, I don't want to make NeuralNote specific to one or a few DAWs that support this feature.

[Yzrsah]

What year is this, 2003? It's been decades since I used a DAW that didn't support plugins with Audio IN + MIDI output. Maybe you got wrong information because you looked at some weird obscure FOSS DAW that didn't support it? I use 5 different DAWs and they all support it.

It's not "make NeuralNote specific to one or a few DAWs that support this feature"
It's "make NeuralNote support standard functionality that is standard for almost all DAWs"

The DAW that can't support it would be the exception. Which DAW doesn't support it?

[mickaelistria]

The inability for most DAWs to have audio in and midi out is only one of the reasons why we can't stream the midi out, even with some large delay.

I can't tell for other possible counter-examples, but at least I can confirm that Carla does support it well, and some other VST sharing similar goals, such as Dodo Music, leverage it well. Here is a screenshot of how this can be easily plugged in an audio flow.

I also tried Widi VST, which doesn't directly send Midi notes to the standard events-out, but has an option to create a virtual Midi device on the machine, and transmits the Midi events through this virtual device (which can then be wired to a synth).

[DamRsn]

After a bit of digging, it seems like the VST3 format supports it but that AU format does not. So Logic Pro for example does not support audio in / midi out.

My plan is to have an option to either use the internal synth of NeuralNote (recent NeuralNote feature, not in a release yet) or to stream midi out (in a channel to be chosen by the user). Then it would be up to the user to figure out if they can route the generated midi to another virtual instrument in they DAW.

What do you think about this?

[RustoMCSpit]

I
[Submit your developer support request]

@DamRsn heres a list of support links for various daws where you can detail what exactly you need to make this viable, trust me they listen to requests

https://www.bitwig.com/support/

https://help.ableton.com/hc/en-us/requests/new?ticket_form_id=360001694780

https://www.image-line.com/fl-studio-support/#panel-75-0-0-3

https://help.reasonstudios.com/hc/requests/new

https://www.apple.com/feedback/logic-pro/

[RustoMCSpit]

I read in the doc """ Very few DAWs support audio input/MIDI output plugins as far as I know. This is partially why NeuralNote is an Audio FX plugin (audio-to-audio) and that MIDI is exported via drag and drop. """ I use a DAW that seems to allow it (Carla), and I also tried Widi VST which does allow to create a new midi source that Carla detect (as a separate input, detached from the plugin, but still present and still allowing to directly map the midi transcription). If this pattern of creating a new virtual midi input for the plugin output can work, I would recommend to implement it.

carla? youre using linux right? is it stable for you???

[mickaelistria]

My plan is to have an option to [...] stream midi out (in a channel to be chosen by the user). Then it would be up to the user to figure out if they can route the generated midi to another virtual instrument in they DAW.

That sounds very good!

[whjvenyl]

would love that and try it out in reaper!!

[jareddyck]

Would love to get this going in ableton live!

[DamRsn]

NeuralNote now supports streaming MIDI output. MIDI output can now be routed to another synth.