Skip to content

Commit

Permalink
Udpate TensorRT benchmark results (Project-MONAI#647)
Browse files Browse the repository at this point in the history
### Description
Update TensorRT benchmark results according to the latest MONAI version
(1.4.0).

### Status
**Ready**

### Please ensure all the checkboxes:
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Codeformat tests passed locally by running `./runtests.sh
--codeformat`.
- [ ] In-line docstrings updated.
- [ ] Update `version` and `changelog` in `metadata.json` if changing an
existing bundle.
- [ ] Please ensure the naming rules in config files meet our
requirements (please refer to: `CONTRIBUTING.md`).
- [ ] Ensure versions of packages such as `monai`, `pytorch` and `numpy`
are correct in `metadata.json`.
- [ ] Descriptions should be consistent with the content, such as
`eval_metrics` of the provided weights and TorchScript modules.
- [ ] Files larger than 25MB are excluded and replaced by providing
download links in `large_file.yml`.
- [ ] Avoid using path that contains personal information within config
files (such as use `/home/your_name/` for `"bundle_root"`).

Signed-off-by: binliu <binliu@nvidia.com>
  • Loading branch information
binliunls authored Sep 11, 2024
1 parent e1a1096 commit 5b8cb67
Show file tree
Hide file tree
Showing 10 changed files with 18 additions and 13 deletions.
3 changes: 2 additions & 1 deletion models/pathology_nuclei_classification/configs/metadata.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{
"schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json",
"version": "0.1.8",
"version": "0.1.9",
"changelog": {
"0.1.9": "update tensorrt benchmark results",
"0.1.8": "enable tensorrt",
"0.1.7": "update to use monai 1.3.1",
"0.1.6": "set image_only to False",
Expand Down
2 changes: 1 addition & 1 deletion models/pathology_nuclei_classification/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ This bundle supports acceleration with TensorRT. The table below displays the sp

| method | torch_tf32(ms) | torch_amp(ms) | trt_tf32(ms) | trt_fp16(ms) | speedup amp | speedup tf32 | speedup fp16 | amp vs fp16|
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| model computation | 12.06 | 20.57 | 3.23 | 1.48 | 0.59 | 3.73 | 8.15 | 13.90 |
| model computation | 20.47 | 20.57 | 2.49 | 1.48 | 1.00 | 8.22 | 13.83 | 13.90 |
| end2end | 45 | 49 | 18 | 18 | 0.92 | 2.50 | 2.50 | 2.72 |

Where:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{
"schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_hovernet_20221124.json",
"version": "0.2.5",
"version": "0.2.6",
"changelog": {
"0.2.6": "update tensorrt benchmark results",
"0.2.5": "enable tensorrt",
"0.2.4": "update to use monai 1.3.1",
"0.2.3": "remove meta_dict usage",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ This bundle supports acceleration with TensorRT. The table below displays the sp

| method | torch_tf32(ms) | torch_amp(ms) | trt_tf32(ms) | trt_fp16(ms) | speedup amp | speedup tf32 | speedup fp16 | amp vs fp16|
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| model computation | 27.15 | 20.14 | 19.54 | 5.63 | 1.35 | 1.39 | 4.82 | 3.58 |
| model computation | 24.55 | 20.14 | 10.85 | 5.63 | 1.22 | 2.26 | 4.36 | 3.58 |
| end2end | 3451 | 3312 | 1318 | 878 | 1.04 | 2.62 | 3.93 | 3.77 |

Where:
Expand Down
3 changes: 2 additions & 1 deletion models/swin_unetr_btcv_segmentation/configs/metadata.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{
"schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json",
"version": "0.5.5",
"version": "0.5.6",
"changelog": {
"0.5.6": "update tensorrt benchmark results",
"0.5.5": "enable tensorrt",
"0.5.4": "update to use monai 1.3.1",
"0.5.3": "add load_pretrain flag for infer",
Expand Down
4 changes: 2 additions & 2 deletions models/swin_unetr_btcv_segmentation/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,8 @@ The `swin_unetr` bundle supports acceleration with TensorRT. The table below dis

| method | torch_tf32(ms) | torch_amp(ms) | trt_tf32(ms) | trt_fp16(ms) | speedup amp | speedup tf32 | speedup fp16 | amp vs fp16|
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| model computation | 503.1 | 123.77 | 229.85 | 42.87 | 4.06 | 2.19 | 11.74 | 2.89 |
| end2end |5102 | 4895 | 4264 | 2835 | 1.04 | 1.20 | 1.80 | 1.73 |
| model computation | 123.64 | 123.77 | 93.22 | 42.87 | 1.00 | 1.33 | 2.88 | 2.89 |
| end2end | 5102 | 4895 | 2863 | 2835 | 1.04 | 1.78 | 1.80 | 1.73 |

Where:
- `model computation` means the speedup ratio of model's inference with a random input without preprocessing and postprocessing
Expand Down
3 changes: 2 additions & 1 deletion models/vista2d/configs/metadata.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{
"schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20240725.json",
"version": "0.2.5",
"version": "0.2.6",
"changelog": {
"0.2.6": "update tensorrt benchmark results",
"0.2.5": "add tensorrt benchmark results",
"0.2.4": "enable tensorrt inference",
"0.2.3": "update weights link",
Expand Down
4 changes: 2 additions & 2 deletions models/vista2d/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ The `vista2d` bundle supports acceleration with TensorRT. The table below displa

| method | torch_tf32(ms) | torch_amp(ms) | trt_tf32(ms) | trt_fp16(ms) | speedup amp | speedup tf32 | speedup fp16 | amp vs fp16|
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| model computation | 90.11 | 39.68 | 71.7 | 17.32 | 2.27 | 1.26 | 5.20 | 2.29 |
| end2end |1562 | 1903 | 1494 | 1440 | 0.82 | 1.05 | 1.08 | 1.32|
| model computation | 39.72 | 39.68 | 26.13 | 17.32 | 1.00 | 1.52 | 2.29 | 2.29 |
| end2end | 1562 | 1903 | 1494 | 1440 | 0.82 | 1.05 | 1.08 | 1.32|

Where:
- `model computation` means the speedup ratio of model's inference with a random input without preprocessing and postprocessing
Expand Down
3 changes: 2 additions & 1 deletion models/vista3d/configs/metadata.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{
"schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20240725.json",
"version": "0.4.6",
"version": "0.4.7",
"changelog": {
"0.4.7": "update tensorrt benchmark results",
"0.4.6": "add tensorrt benchmark result and remove the metric part",
"0.4.5": "remove wrong path",
"0.4.4": "enable tensorrt inference",
Expand Down
4 changes: 2 additions & 2 deletions models/vista3d/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ The `vista3d` bundle supports acceleration with TensorRT. The table below displa

| method | torch_tf32(ms) | torch_amp(ms) | trt_tf32(ms) | trt_fp16(ms) | speedup amp | speedup tf32 | speedup fp16 | amp vs fp16|
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| model computation | 577.00 | 91.90 | 353.69 | 60.02 | 6.28 | 1.63 | 9.58 | 1.53 |
| end2end |6740 | 5166 | 5242 | 3386 | 1.30 | 1.29 | 1.99 | 1.53 |
| model computation | 108.53| 91.9 | 106.84 | 60.02 | 1.18 | 1.02 | 1.81 | 1.53 |
| end2end | 6740 | 5166 | 5242 | 3386 | 1.30 | 1.29 | 1.99 | 1.53 |

Where:
- `model computation` means the speedup ratio of model's inference with a random input without preprocessing and postprocessing
Expand Down

0 comments on commit 5b8cb67

Please sign in to comment.