How to Convert Halpe26 to TensorRT Format

[NetxueGmail]

When I use the following code to convert to TensorRT format, it runs without any issues:

python tools/deploy.py
configs/mmpose/pose-detection_simcc_tensorrt_dynamic-256x192.py
../mmpose/projects/rtmpose/rtmpose/body_2d_keypoint/rtmpose-m_8xb256-420e_coco-256x192.py
https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-aic-coco_pt-aic-coco_420e-256x192-63eb25f7_20230126.pth
demo/resources/human-pose.jpg
--work-dir rtmpose-trt/rtmpose-m
--device cuda:0
--show
--dump-info

However, when I switch to the Halpe26 model and attempt to convert it to the TensorRT FP16 format, errors occur after running the following code:

python tools/deploy.py
configs/mmpose/pose-detection_tensorrt-fp16_static-256x192.py
../mmpose/projects/rtmpose/rtmpose/body_2d_keypoint/rtmpose-m_8xb512-700e_body8-halpe26-256x192.py
/home/mocap/Documents/MMPose/checkpoints/rtmpose-m_simcc-body7_pt-body7-halpe26_700e-256x192-4d3e73dd_20230605.pth
demo/resources/human-pose.jpg
--work-dir rtmpose-trt/rtmpose-m
--device cuda:0
--show
--dump-info

The error details are as follows:

12/25 14:53:11 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
12/25 14:53:11 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
12/25 14:53:12 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
12/25 14:53:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
12/25 14:53:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: /home/mocap/Documents/MMPose/checkpoints/rtmpose-m_simcc-body7_pt-body7-halpe26_700e-256x192-4d3e73dd_20230605.pth
12/25 14:53:13 - mmengine - WARNING - configs/base/datasets/halpe26.py is not a meta file, simply parsed as meta information
12/25 14:53:13 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
12/25 14:53:13 - mmengine - INFO - Export PyTorch model to ONNX: rtmpose-trt/rtmpose-m/end2end.onnx.
12/25 14:53:13 - mmengine - WARNING - Can not find torch.nn.functional._scaled_dot_product_attention, function rewrite will not be applied
12/25 14:53:13 - mmengine - WARNING - Can not find mmdet.models.utils.transformer.PatchMerging.forward, function rewrite will not be applied
12/25 14:53:14 - mmengine - INFO - Execute onnx optimize passes.
12/25 14:53:14 - mmengine - WARNING - Can not optimize model, please build torchscipt extension.
More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md
12/25 14:53:14 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
12/25 14:53:16 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in subprocess
12/25 14:53:16 - mmengine - WARNING - Could not load the library of tensorrt plugins. Because the file does not exist:
[12/25/2024-14:53:16] [TRT] [I] [MemUsageChange] Init CUDA: CPU +13, GPU +0, now: CPU 123, GPU 936 (MiB)
[12/25/2024-14:53:19] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +1449, GPU +266, now: CPU 1649, GPU 1202 (MiB)
[12/25/2024-14:53:19] [TRT] [I] ----------------------------------------------------------------
[12/25/2024-14:53:19] [TRT] [I] Input filename: rtmpose-trt/rtmpose-m/end2end.onnx
[12/25/2024-14:53:19] [TRT] [I] ONNX IR version: 0.0.6
[12/25/2024-14:53:19] [TRT] [I] Opset version: 11
[12/25/2024-14:53:19] [TRT] [I] Producer name: pytorch
[12/25/2024-14:53:19] [TRT] [I] Producer version: 2.3.1
[12/25/2024-14:53:19] [TRT] [I] Domain:
[12/25/2024-14:53:19] [TRT] [I] Model version: 0
[12/25/2024-14:53:19] [TRT] [I] Doc string:
[12/25/2024-14:53:19] [TRT] [I] ----------------------------------------------------------------
[12/25/2024-14:53:19] [TRT] [W] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[12/25/2024-14:53:19] [TRT] [I] Graph optimization time: 0.0160444 seconds.
[12/25/2024-14:53:19] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[12/25/2024-14:56:48] [TRT] [I] Detected 1 inputs and 2 output network tensors.
[12/25/2024-14:56:48] [TRT] [I] Total Host Persistent Memory: 407664
[12/25/2024-14:56:48] [TRT] [I] Total Device Persistent Memory: 1024
[12/25/2024-14:56:48] [TRT] [I] Total Scratch Memory: 0
[12/25/2024-14:56:48] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 33 MiB, GPU 28 MiB
[12/25/2024-14:56:48] [TRT] [I] [BlockAssignment] Started assigning block shifts. This will take 169 steps to complete.
[12/25/2024-14:56:48] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 3.42546ms to assign 8 blocks to 169 nodes requiring 2373632 bytes.
[12/25/2024-14:56:48] [TRT] [I] Total Activation Memory: 2372608
[12/25/2024-14:56:48] [TRT] [W] TensorRT encountered issues when converting weights between types and that could affect accuracy.
[12/25/2024-14:56:48] [TRT] [W] If this is not the desired behavior, please modify the weights or retrain with regularization to adjust the magnitude of the weights.
[12/25/2024-14:56:48] [TRT] [W] Check verbose logs for the list of affected weights.
[12/25/2024-14:56:48] [TRT] [W] - 57 weights are affected by this issue: Detected subnormal FP16 values.
[12/25/2024-14:56:48] [TRT] [W] - 19 weights are affected by this issue: Detected values less than smallest positive FP16 subnormal value and converted them to the FP16 minimum subnormalized value.
[12/25/2024-14:56:48] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +26, GPU +28, now: CPU 26, GPU 28 (MiB)
12/25 14:56:48 - mmengine - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend
12/25 14:56:48 - mmengine - INFO - visualize tensorrt model start.
12/25 14:56:50 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
12/25 14:56:50 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
12/25 14:56:50 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "backend_segmentors" registry tree. As a workaround, the current "backend_segmentors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
12/25 14:56:50 - mmengine - WARNING - Could not load the library of tensorrt plugins. Because the file does not exist:
12/25 14:56:50 - mmengine - WARNING - configs/base/datasets/halpe26.py is not a meta file, simply parsed as meta information
2024-12-25:14:56:50 - root - ERROR - not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
File "/home/mocap/mmdeploy/mmdeploy/utils/utils.py", line 41, in target_wrapper
result = target(*args, **kwargs)
File "/home/mocap/mmdeploy/mmdeploy/apis/visualize.py", line 72, in visualize_model
result = model.test_step(model_inputs)[0]
File "/home/mocap/anaconda3/envs/work/lib/python3.10/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
return self._run_forward(data, mode='predict') # type: ignore
File "/home/mocap/anaconda3/envs/work/lib/python3.10/site-packages/mmengine/model/base_model/base_model.py", line 361, in _run_forward
results = self(**data, mode=mode)
File "/home/mocap/anaconda3/envs/work/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/mocap/anaconda3/envs/work/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/home/mocap/mmdeploy/mmdeploy/codebase/mmpose/deploy/pose_detection_model.py", line 117, in forward
batch_pred_x, batch_pred_y = batch_outputs
ValueError: not enough values to unpack (expected 2, got 1)
[12/25/2024-14:56:50] [TRT] [E] 1: [defaultAllocator.cpp::deallocate::61] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[12/25/2024-14:56:50] [TRT] [E] 1: [defaultAllocator.cpp::deallocate::61] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[12/25/2024-14:56:50] [TRT] [E] 1: [defaultAllocator.cpp::deallocate::61] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[12/25/2024-14:56:50] [TRT] [E] 1: [cudaResources.cpp::~ScopedCudaStream::47] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[12/25/2024-14:56:50] [TRT] [E] 1: [cudaResources.cpp::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
.
.
.
.
.
.
[cudaDriverHelpers.cpp::operator()::94] Error Code 1: Cuda Driver (an illegal memory access was encountered)
[12/25/2024-14:56:51] [TRT] [E] 1: [cudaDriverHelpers.cpp::operator()::94] Error Code 1: Cuda Driver (an illegal memory access was encountered)
12/25 14:56:52 - mmengine - ERROR - /home/mocap/mmdeploy/tools/deploy.py - create_process - 82 - visualize tensorrt model failed.

Please help me understand the cause of this issue.Thanks very much