Set specific Cuda stream ID in Cpp implimentation code.

[phamthanhdat270198]

Env

• Agx Orin 64GB.
• Ubuntu20.04
• Cuda version 11.4
• TensorRT version 8.5.2

Question

Hi,
I want to set specific cudastream ID on Cpp code yolo like --streams=0 or --streams=1, ... in trtexec
Many thanks!!!!!

[wang-xinyu]

You need to adapt the inference code, create multiple streams and pass to infer() function.

[phamthanhdat270198]

When I create multiple streams It just decrease a little bit infer time. So Does it working in parallel. give me an example about multi streams and multi contexts. Many thanks!!!
Here is my impliment multi streams