Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/quantizer factory refactoring #1963

Draft
wants to merge 62 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
e6bc8fe
OpenVINO
BloodAxe Mar 28, 2024
246d59d
OpenVINO
BloodAxe Mar 28, 2024
e284b31
PTQ with QC
BloodAxe Mar 28, 2024
05c38fa
OpenVINO
BloodAxe Mar 28, 2024
b27ce4d
quantize_with_accuracy_control
BloodAxe Mar 28, 2024
83dd059
coco2017_yolo_nas_s_ptq_only
BloodAxe Mar 28, 2024
d33a1eb
coco2017_yolo_nas_s_ptq_only
BloodAxe Mar 28, 2024
96f3049
coco2017_yolo_nas_s_ptq_only
BloodAxe Mar 28, 2024
f0536d4
coco2017_yolo_nas_s_ptq_only_cpu
BloodAxe Mar 28, 2024
e0e5c91
coco2017_yolo_nas_s_ptq_only_cpu
BloodAxe Mar 29, 2024
08b829e
coco2017_yolo_nas_s_ptq_only_cpu
BloodAxe Mar 29, 2024
93c507c
coco2017_yolo_nas_s_ptq_only_cpu
BloodAxe Mar 29, 2024
10146bc
coco2017_yolo_nas_s_ptq_only_cpu
BloodAxe Mar 29, 2024
ec135ca
coco2017_yolo_nas_s_ptq_only_cpu
BloodAxe Mar 29, 2024
2aba1cb
coco2017_yolo_nas_s_ptq_only_cpu
BloodAxe Mar 29, 2024
eb75e0f
Working PoC of export of quantized model
BloodAxe Apr 2, 2024
240c0b1
PoC of the plugin-based exporter & quantizer
BloodAxe Apr 4, 2024
792aaf5
PoC of the plugin-based exporter & quantizer
BloodAxe Apr 5, 2024
22ead7b
Make TRT & Vino quantizers work for PTQ
BloodAxe Apr 8, 2024
faeaf70
Make TRT QAT work
BloodAxe Apr 9, 2024
0a8b01d
Implementing export (WIP)
BloodAxe Apr 9, 2024
a2fd101
Update exporters API
BloodAxe Apr 10, 2024
0f5f8ca
Update exporters API
BloodAxe Apr 10, 2024
04e9230
Print only common metircs
BloodAxe Apr 10, 2024
a701f85
Print only common metircs
BloodAxe Apr 11, 2024
72e590d
Tune QAT parameters
BloodAxe Apr 11, 2024
7a42947
Split PTQ & QAT methods into two different classes
BloodAxe Apr 11, 2024
be785d9
Added doc
BloodAxe Apr 11, 2024
bc1741b
Fix qat using wrong model
BloodAxe Apr 11, 2024
fdc25cd
Update trainer
BloodAxe Apr 12, 2024
c5301d4
Fix YoloNAS Pose head support for OpenVINO PTQ
BloodAxe Apr 12, 2024
cb404ba
Bump up package version and remove OpenVINO deps
BloodAxe Apr 15, 2024
6dd186d
Merge branch 'refs/heads/master' into feature/openvino-refactoring
BloodAxe Apr 15, 2024
519cbe9
Merge branch 'refs/heads/master' into feature/openvino-refactoring
BloodAxe Apr 15, 2024
173f00d
Remove openvino stuff
BloodAxe Apr 15, 2024
e8bb9aa
Remove openvino stuff
BloodAxe Apr 15, 2024
63f6d3f
Remove openvino stuff
BloodAxe Apr 15, 2024
fa2eb86
Revert irrelevant stuff
BloodAxe Apr 15, 2024
0d111e9
Revert irrelevant stuff
BloodAxe Apr 15, 2024
cbc670f
postprocessing_use_tensorrt_nms -> detection_postprocessing_use_tenso…
BloodAxe Apr 15, 2024
d708ae1
Revert src/super_gradients/modules/skip_connections.py
BloodAxe Apr 15, 2024
24e1355
Remove irrelevant stuff
BloodAxe Apr 15, 2024
6b7b19e
Update QATRecipeModificationCallback
BloodAxe Apr 15, 2024
65f174c
Adding back ptq() and qat() methods
BloodAxe Apr 16, 2024
54c8b58
Adding back ptq() and qat() methods
BloodAxe Apr 16, 2024
a8ab948
Force PTQ/QAT modes when called from corresponding methods and legacy…
BloodAxe Apr 16, 2024
8854ab6
Improve output path handling of exported model
BloodAxe Apr 17, 2024
58ff42b
Remove TODO
BloodAxe Apr 17, 2024
4d14410
Remove leftover
BloodAxe Apr 17, 2024
39416ae
Added missing max_batches docstring
BloodAxe Apr 17, 2024
9d1fe2b
Update notebook
BloodAxe Apr 18, 2024
77427bc
Added missing installation pf trt quantizer for
BloodAxe Apr 18, 2024
20423af
Reoranize quantizers sub-package to allow lazy-install of pytorch-qua…
BloodAxe Apr 18, 2024
8974d56
Fix tests
BloodAxe Apr 18, 2024
f89f9ba
Fix syntax error
BloodAxe Apr 18, 2024
5836dae
Fix import
BloodAxe Apr 18, 2024
aa3264f
Fix import
BloodAxe Apr 18, 2024
be70f48
Fix import
BloodAxe Apr 18, 2024
5ecefd1
Update docs in YAML files
BloodAxe Apr 18, 2024
a6cf4d3
Update notebook to use postprocessing_use_tensorrt_nms:bool instead o…
BloodAxe Apr 19, 2024
0a3c076
Update notebook to use postprocessing_use_tensorrt_nms:bool instead o…
BloodAxe Apr 19, 2024
8be4575
Fixing tests
BloodAxe Apr 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions documentation/source/ptq_qat.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,17 +259,14 @@ If you prefer more of a DIY approach, here is the code sample:

```python
import torch
from super_gradients.training.utils.quantization.export import export_quantized_module_to_onnx

from super_gradients.conversion.onnx import export_to_onnx
onnx_filename = f"qat_model_1x3x224x224.onnx"

dummy_input = torch.randn([1, 3, 224, 224], device="cpu")
export_quantized_module_to_onnx(
model=quantized_model.cpu(),
export_to_onnx(
model=quantized_model.eval(),
model_input=dummy_input,
onnx_filename=onnx_filename,
input_shape=[1, 3, 224, 224],
input_size=[1, 3, 224, 224],
train=False,
)
```

Expand Down
2 changes: 1 addition & 1 deletion notebooks/DEKR_PoseEstimationFineTuning.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
{
"cell_type": "code",
"source": [
"!pip install -q super_gradients==3.7.1 scikit-learn"
"!pip install -q super_gradients==3.8.0 scikit-learn"
],
"metadata": {
"id": "3UZJqTehg0On",
Expand Down
2 changes: 1 addition & 1 deletion notebooks/PTQ_and_QAT_for_classification.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
},
"outputs": [],
"source": [
"!pip -qq install super-gradients==3.7.1 pytorch-quantization==2.1.2 --extra-index-url https://pypi.ngc.nvidia.com"
"!pip -qq install super-gradients==3.8.0 pytorch-quantization==2.1.2 --extra-index-url https://pypi.ngc.nvidia.com"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/Segmentation_Model_Export.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@
"execution_count": 30,
"outputs": [],
"source": [
"!pip install -qq super_gradients==3.7.1"
"!pip install -qq super_gradients==3.8.0"
],
"metadata": {
"colab": {
Expand Down
8 changes: 4 additions & 4 deletions notebooks/YoloNAS_Inference_using_TensorRT.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@
"# This hack is to prevent occasional issue in Colab https://github.com/googlecolab/colabtools/issues/3409\n",
"locale.getpreferredencoding = lambda *args: \"UTF-8\"\n",
"\n",
"!pip install -qq super-gradients==3.7.1\n",
"!pip install -qq super-gradients==3.8.0\n",
"!pip install -qq tensorrt~=8.6 pycuda pytorch-quantization==2.1.2 --extra-index-url https://pypi.ngc.nvidia.com"
],
"metadata": {
Expand Down Expand Up @@ -337,7 +337,7 @@
"from super_gradients.conversion.conversion_enums import ExportTargetBackend, ExportQuantizationMode, DetectionOutputFormatMode\n",
"\n",
"yolonas = models.get(Models.YOLO_NAS_M, pretrained_weights=\"coco\")\n",
"yolonas.export(\"yolonas_m.onnx\", preprocessing=True, postprocessing=True, engine=ExportTargetBackend.TENSORRT)"
"yolonas.export(\"yolonas_m.onnx\", preprocessing=True, postprocessing=True, postprocessing_use_tensorrt_nms=True)"
],
"metadata": {
"colab": {
Expand Down Expand Up @@ -647,7 +647,7 @@
"def convert_onnx_to_trt_engine(onnx_file, trt_output_file, enable_int8_quantization:bool = False):\n",
" EXPLICIT_BATCH = 1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)\n",
"\n",
" with trt.Builder(trt_logger) as builder, builder.create_network(EXPLICIT_BATCH) as network, builder.create_builder_config() as config:\n",
" with trt.Builder(trt_logger) as builder, builder.create_network(EXPLICIT_BATCH) as network:\n",
"\n",
" config = builder.create_builder_config()\n",
" config.profiling_verbosity = trt.ProfilingVerbosity.DETAILED\n",
Expand Down Expand Up @@ -1068,7 +1068,7 @@
"from super_gradients.conversion.conversion_enums import ExportQuantizationMode\n",
"\n",
"yolonas = models.get(Models.YOLO_NAS_M, pretrained_weights=\"coco\")\n",
"yolonas.export(\"yolonas_m_int8.onnx\", preprocessing=True, postprocessing=True, engine=ExportTargetBackend.TENSORRT, quantization_mode=ExportQuantizationMode.INT8)\n",
"yolonas.export(\"yolonas_m_int8.onnx\", preprocessing=True, postprocessing=True, postprocessing_use_tensorrt_nms=True, quantization_mode=ExportQuantizationMode.INT8)\n",
"convert_onnx_to_trt_engine(\"yolonas_m_int8.onnx\", \"yolonas_m_int8.trt\", enable_int8_quantization=True)"
],
"metadata": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@
},
"outputs": [],
"source": [
"!pip install -qq super-gradients==3.7.1 gdown"
"!pip install -qq super-gradients==3.8.0 gdown"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/albumentations_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@
}
],
"source": [
"! pip install -qq super-gradients==3.7.1\n",
"! pip install -qq super-gradients==3.8.0\n",
"! pip install -qq albumentations == 1.3.1\n",
"! pip install -qq datasets[vision]~=2.1 pandas~=2.0"
]
Expand Down
2 changes: 1 addition & 1 deletion notebooks/dataloader_adapter.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
{
"cell_type": "code",
"source": [
"!pip install -q super-gradients==3.7.1"
"!pip install -q super-gradients==3.8.0"
],
"metadata": {
"id": "0puCRQGZSP8r",
Expand Down
2 changes: 1 addition & 1 deletion notebooks/detection_how_to_connect_custom_dataset.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@
},
"outputs": [],
"source": [
"! pip install -q super_gradients==3.7.1"
"! pip install -q super_gradients==3.8.0"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/detection_transfer_learning.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@
}
],
"source": [
"! pip install -qq super_gradients==3.7.1"
"! pip install -qq super_gradients==3.8.0"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/how_to_run_model_predict.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
}
],
"source": [
"!pip install -qq super-gradients==3.7.1"
"!pip install -qq super-gradients==3.8.0"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
},
"outputs": [],
"source": [
"!pip install -qq super_gradients==3.7.1"
"!pip install -qq super_gradients==3.8.0"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/quickstart_segmentation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
},
"outputs": [],
"source": [
"! pip install -qq super-gradients==3.7.1"
"! pip install -qq super-gradients==3.8.0"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/segmentation_connect_custom_dataset.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@
}
],
"source": [
"! pip install -qq super-gradients==3.7.1"
"! pip install -qq super-gradients==3.8.0"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/transfer_learning_classification.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
}
],
"source": [
"! pip install -qq super-gradients==3.7.1"
"! pip install -qq super-gradients==3.8.0"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/transfer_learning_semantic_segmentation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
},
"outputs": [],
"source": [
"! pip install -qq super-gradients==3.7.1"
"! pip install -qq super-gradients==3.8.0"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/what_are_recipes_and_how_to_use.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
{
"cell_type": "code",
"source": [
"!pip install -q super-gradients==3.7.1"
"!pip install -q super-gradients==3.8.0"
],
"metadata": {
"id": "8uZM-4va5Rpu",
Expand Down
Loading
Loading