Convert ONNX to ENGINE file failure of TensorRT 10.0.0.6 when running trtexec on GPU NVIDIA ORIN NX 16GB

[robin-ai-ml]

Description

I tried to use trtexec to convert the onnx model to engine file but fails with the error below

[04/19/2024-14:04:31] [V] [TRT] =============== Computing costs for /patch_embed/proj/Conv
[04/19/2024-14:04:31] [V] [TRT] *************** Autotuning format combination: Float((* 3 E0),E0,width,1) where E0=(* height width) -> Float((* 384 E1),E1,E0,1) where E0=(+ (CEIL_DIV (+ width -27) 14) 1) E1=(* (+ (CEIL_DIV (+ height -27) 14) 1) E0) where E0=(* height width) E1=(+ (CEIL_DIV (+ width -27) 14) 1) E2=(* (+ (CEIL_DIV (+ height -27) 14) 1) E1) ***************
[04/19/2024-14:04:31] [V] [TRT] --------------- Timing Runner: /patch_embed/proj/Conv (CaskConvolution[0x80000009])
[04/19/2024-14:04:31] [E] Error[9]: Skipping tactic 0x5deb29b7a8e275f7 due to exception initDeviceReservedSpace
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0xa8ef60e712f8ad24 due to exception initDeviceReservedSpace
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0x31aa67f57c5aea77 due to exception Cask convolution execution
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0xa9366041633a5135 due to exception Cask convolution execution
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0xede36641840ce3d2 due to exception Cask convolution execution
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0x40a12e3938221818 due to exception Cask convolution execution
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0xcb8a43f748d8a338 due to exception Cask convolution execution
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0x9808072e706def96 due to exception initDeviceReservedSpace
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0xd828f024626fa982 due to exception Cask convolution execution
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0x1673e3594ce11cea due to exception Cask convolution execution
[04/19/2024-14:04:32] [E] Error[9]: Skipping tactic 0xc3cf6e1d1c6aff27 due to exception initDeviceReservedSpace

....................................ignore more erro lines .......................................

....................................ignore more erro lines .......................................

[04/19/2024-14:11:30] [E] Error[10]: [optimizer.cpp::computeCosts::4103] Error Code 10: Internal Error (Could not find any implementation for node /patch_embed/proj/Conv.)
[04/19/2024-14:11:30] [E] Error[1]: [checkMacros.cpp::catchCudaError::181] Error Code 1: Cuda Runtime (no kernel image is available for execution on the device)
[04/19/2024-14:11:30] [E] Engine could not be created from network
[04/19/2024-14:11:30] [E] Building engine failed
[04/19/2024-14:11:30] [E] Failed to create engine from model or file.
[04/19/2024-14:11:30] [E] Engine set up failed

Environment

Operating system Version: Ubuntu 22.04

TensorRT Version: 10.0.06

NVIDIA GPU: NVIDIA ORIN NX 16GB

NVIDIA Driver Version:

CUDA Version: 12.4.131

CUDNN Version: 9.1.0

Operating System:

Python Version (if applicable): 3.10.12

Tensorflow Version (if applicable): X

PyTorch Version (if applicable): X

Baremetal or Container (if so, version): X

Relevant Files

Model link: https://github.com/fabio-sim/Depth-Anything-ONNX/releases/download/v1.0.0/depth_anything_vits14.onnx

Steps To Reproduce

Commands or scripts:
/usr/src/tensorrt/bin/trtexec --verbose --best --onnx=depth_anything_vits14.onnx --saveEngine=vits14.engine --minShapes=image:1x3x128x128 --optShapes=image:1x3x518x518 --maxShapes=image:1x3x640x960

Have you tried the latest release?: it is already the latest tesorrt version 10.0

Can this model run on other frameworks? this onnx can run on an RTX3090 with Python infererence

[robin-ai-ml]

here is complete detail error information Uploading bug_report.txt…

[lix19937]

Cuda Runtime (no kernel image is available for execution on the device)

Maybe this is root cause.
It means that there is no binary for your GPU card. Usually, you should test whether CUDA is available in the current environment first.

If this onnx can run on an RTX3090 with Python infererence, you should make sure two version of tensorrt are the same. Because the trtexec(binary) is not build by yourself.

[zerollzeng]

Wait... How do you install TensorRT 10 in NX? we haven't release the corresponding Jetpack/DriveOS?

[ttyio]

closing since no activity for more than 3 weeks, pls reopen if you still have question, thanks all!