Update to NvAIE 3.0 PYDS 1.1.5

This release is compatible with DeepStream SDK 6.2 Ubuntu 20.04 Python 3.8 DeepStream SDK 6.2 Features: - New app deepstream-imagedata-multistream-cupy added - New app deepstream-segmask added - New app deepstream-custom-binding-test added - New bindings guide now available along with custom bindings guide - Updated deepstream-test1 notebook and app - New bindings added: unmap_nvds_buf_surface() - Updated deepstream-imagedata-multistream and deepstream-imagedata-multistream-redaction to use the above new binding
NVIDIA-AI-IOT · Dec 15, 2022 · 883c0e3 · 883c0e3
1 parent f70dcc9
commit 883c0e3
Show file tree

Hide file tree

Showing 124 changed files with 5,597 additions and 524 deletions.
diff --git a/FAQ.md b/FAQ.md
@@ -1,15 +1,42 @@
 # Frequently Asked Questions and Troubleshooting Guide
 
-* [Git clone fails due to Gstreamer repo access error](#faq9)
-* [Application fails to work with mp4 stream](#faq0)  
-* [Ctrl-C does not stop the app during engine file generation](#faq1)  
-* [Application fails to create gst elements](#faq2)  
-* [GStreamer debugging](#faq3)  
-* [Application stuck with no playback](#faq4)  
-* [Error on setting string field](#faq5)  
-* [Pipeline unable to perform at real time](#faq6)  
-* [Triton container problems with multi-GPU setup](#faq7)
-* [ModuleNotFoundError: No module named 'pyds'](#faq8)
+- [Frequently Asked Questions and Troubleshooting Guide](#frequently-asked-questions-and-troubleshooting-guide)
+    - [Using new gst-nvstreammux](#using-new-gst-nvstreammux)
+    - [Git clone fails due to Gstreamer repo access error](#git-clone-fails-due-to-gstreamer-repo-access-error)
+    - [Application fails to work with mp4 stream](#application-fails-to-work-with-mp4-stream)
+    - [Ctrl-C does not stop the app during engine file generation](#ctrl-c-does-not-stop-the-app-during-engine-file-generation)
+    - [Application fails to create gst elements](#application-fails-to-create-gst-elements)
+    - [GStreamer debugging](#gstreamer-debugging)
+    - [Application stuck with no playback](#application-stuck-with-no-playback)
+    - [Error on setting string field](#error-on-setting-string-field)
+    - [Pipeline unable to perform at real time](#pipeline-unable-to-perform-at-real-time)
+    - [Triton container problems with multi-GPU setup](#triton-container-problems-with-multi-gpu-setup)
+    - [ModuleNotFoundError: No module named 'pyds'](#modulenotfounderror-no-module-named-pyds)
+
+<a name="faq10"></a>
+### Using new gst-nvstreammux
+Most DeepStream Python apps are written to use the default nvstreammux and have lines of code written to set nvstreammux properties which are deprecated in the new gst-nvstreammux. See the [DeepStream documentation](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvstreammux2.html) for more information on the new gst-nvstreammux plugin. To use the new nvstreammux, set the `USE_NEW_NVSTREAMMUX` environment variable before running the app. For example:
+```bash
+   $ export USE_NEW_NVSTREAMMUX="yes"
+   $ python3 deepstream_test_1.py ../../../../samples/streams/sample_720p.h264
+```
+The app itself must be modified not to set deprecated properties when using the new nvstreammux. See [deepstream-test1](./apps/deepstream-test1/deepstream_test_1.py):
+```python
+if os.environ.get('USE_NEW_NVSTREAMMUX') != 'yes': # Only set these properties if not using new gst-nvstreammux
+  streammux.set_property('width', 1920)
+  streammux.set_property('height', 1080)
+  streammux.set_property('batched-push-timeout', 4000000)
+```
+
+Running apps without this modification will result in such an error:
+```
+Traceback (most recent call last):
+  File "deepstream_test_1.py", line 255, in <module>
+    sys.exit(main(sys.argv))
+  File "deepstream_test_1.py", line 194, in main
+    streammux.set_property('width', 1920)
+TypeError: object of type `GstNvStreamMux' does not have property `width'
+```
 
 <a name="faq9"></a>
 ### Git clone fails due to Gstreamer repo access error
@@ -43,9 +70,9 @@ https://docs.python.org/3/library/signal.html
 To work around this:  
 1. Use ctrl-z to bg the process  
 2. Optionally run "jobs" if there are potentially multiple bg processes:  
-    $ jobs  
-    [1]-  Stopped                 python3 deepstream_test_1.py /opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_720p.h264  (wd: /opt/nvidia/deepstream/deepstream-4.0/sources/apps/python/deepstream-test1)  
-    [2]+  Stopped                 python3 deepstream_test_2.py /opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_720p.h264  
+    $ jobs
+    [1]-  Stopped                 python3 deepstream_test_1.py /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264  (wd: /opt/nvidia/deepstream/deepstream-4.0/sources/apps/python/deepstream-test1)
+    [2]+  Stopped                 python3 deepstream_test_2.py /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264
 3. Kill the bg job:  
     $ kill %<job number, 1 or 2 from above. e.g. kill %1>  
 
@@ -125,5 +152,5 @@ The pyds wheel installs the pyds.so library where all the pip packages are store
 
 Command to install the pyds wheel is:
 ```bash
-   $ pip3 install ./pyds-1.1.0-py3-none*.whl
-```
+   $ pip3 install ./pyds-1.1.5-py3-none*.whl
+```
diff --git a/HOWTO.md b/HOWTO.md
@@ -16,7 +16,7 @@ This guide provides resources for DeepStream application development in Python.
 ## Prerequisites
 
 * Ubuntu 20.04
-* [DeepStream SDK 6.1.1](https://developer.nvidia.com/deepstream-download) or later
+* [DeepStream SDK 6.2](https://developer.nvidia.com/deepstream-download) or later
 * Python 3.8
 * [Gst Python](https://gstreamer.freedesktop.org/modules/gst-python.html) v1.16.2
 
@@ -46,11 +46,11 @@ Note: Compiling bindings now also generates a pip installable python wheel for t
 <a name="run_samples"></a>
 ## Running Sample Applications
 
-Clone the deepstream_python_apps repo under <DeepStream 6.1.1 ROOT>/sources:
+Clone the deepstream_python_apps repo under <DeepStream 6.2 ROOT>/sources:
 git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps
 
 This will create the following directory:  
-```<DeepStream 6.1.1 ROOT>/sources/deepstream_python_apps```
+```<DeepStream 6.2 ROOT>/sources/deepstream_python_apps```
 
 The Python apps are under the "apps" directory.  
 Go into each app directory and follow instructions in the README.  
@@ -212,5 +212,5 @@ This function populates the input buffer with a timestamp generated according to
 <a name="imagedata_access"></a>
 ## Image Data Access
 
-Decoded images are accessible as NumPy arrays via the `get_nvds_buf_surface` function. This function is documented in the [API Guide](https://docs.nvidia.com/metropolis/deepstream/6.0.1/python-api/index.html).
+Decoded images are accessible as NumPy arrays via the `get_nvds_buf_surface` function. This function is documented in the [API Guide](https://docs.nvidia.com/metropolis/deepstream/dev-guide/python-api/index.html).
 Please see the [deepstream-imagedata-multistream](apps/deepstream-imagedata-multistream) sample application for an example of image data usage.
diff --git a/README.md b/README.md
@@ -2,11 +2,15 @@
 
 This repository contains Python bindings and sample applications for the [DeepStream SDK](https://developer.nvidia.com/deepstream-sdk).  
 
-SDK version supported: 6.1.1
+<b>NOTE: This branch is for NVAIE 3.0 support only.<b> 
 
-<b>The bindings sources along with build instructions are now available under [bindings](bindings)!  </b>
+<b>Support<b>  
+SDK version:    6.2  
+OS version:     Ubuntu 20.04  
+Python version: 3.8  
+Gst-Python version: [1.16.2](3rdparty/gst-python/)
 
-<b>This release comes with Operating System upgrades (from Ubuntu 18.04 to Ubuntu 20.04) for DeepStreamSDK 6.1.1 support. This translates to upgrade in Python version to 3.8 and [gst-python](3rdparty/gst-python/) version has also been upgraded to 1.16.2 !</b>
+<b>The bindings sources along with build instructions are now available under [bindings](bindings)! We now include a [guide](bindings/BINDINGSGUIDE.md) for adding bindings and another [guide](bindings/CUSTOMUSERMETAGUIDE.md) for advanced use-cases such as writing bindings for custom data structures.</b>
 
 Download the latest release package complete with bindings and sample applications from the [release section](../../releases).  
 
@@ -56,7 +60,8 @@ We currently provide the following sample applications:
 * [deepstream-imagedata-multistream-redaction](apps/deepstream-imagedata-multistream-redaction) -- multi-stream pipeline with face detection and redaction
 * [deepstream-rtsp-in-rtsp-out](apps/deepstream-rtsp-in-rtsp-out) -- multi-stream pipeline with RTSP input/output
 * [deepstream-preprocess-test](apps/deepstream-preprocess-test) -- multi-stream pipeline using nvdspreprocess plugin with custom ROIs
-* <b>NEW</b> [deepstream-demux-multi-in-multi-out](apps/deepstream-demux-multi-in-multi-out) -- multi-stream pipeline using nvstreamdemux plugin to generated separate buffer outputs
+* [deepstream-demux-multi-in-multi-out](apps/deepstream-demux-multi-in-multi-out) -- multi-stream pipeline using nvstreamdemux plugin to generated separate buffer outputs
+* <b>NEW</b> [deepstream-imagedata-multistream-cupy](apps/deepstream-imagedata-multistream-cupy) -- access imagedata buffer from GPU in a multistream source as CuPy array - x86 only
 
 
 Detailed application information is provided in each application's subdirectory under [apps](apps).  

diff --git a/apps/README b/apps/README
@@ -20,7 +20,7 @@ DeepStream SDK Python Bindings
 ================================================================================
 Setup pre-requisites:
 - Ubuntu 20.04
-- NVIDIA DeepStream SDK 6.1.1
+- NVIDIA DeepStream SDK 6.2
 - Python 3.8
 - Gst-python
 
@@ -47,12 +47,15 @@ Package Contents
    deepstream-nvdsanalytics
    deepstream-preprocess-test
    runtime_source_add_delete
+   deepstream-demux-multi-in-multi-out
+   deepstream-imagedata-multistream-cupy
+
 
 --------------------------------------------------------------------------------
 Installing Pre-requisites:
 --------------------------------------------------------------------------------
 
-DeepStream SDK 6.1.1
+DeepStream SDK 6.2
 --------------------
 Download and install from https://developer.nvidia.com/deepstream-download
 
@@ -70,7 +73,7 @@ $ sudo apt install python3-gi python3-dev python3-gst-1.0 -y
 --------------------------------------------------------------------------------
 Running the samples
 --------------------------------------------------------------------------------
-The apps are configured to work from inside the DeepStream SDK 6.1.1 installation.
+The apps are configured to work from inside the DeepStream SDK 6.2 installation.
 
 Clone the deepstream_python_apps repo under <DeepStream ROOT>/sources:
 $ git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps
@@ -109,4 +112,4 @@ Notes:
 --------------------------------------------------------------------------------
 As with DeepStream SDK, if the application runs into errors, cannot create gst elements, 
 try again after removing gstreamer cache
-   rm ${HOME}/.cache/gstreamer-1.0/*
+   rm ${HOME}/.cache/gstreamer-1.0/* 
diff --git a/apps/deepstream-custom-binding-test/README b/apps/deepstream-custom-binding-test/README
@@ -0,0 +1,45 @@
+################################################################################
+# SPDX-FileCopyrightText: Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+Prequisites:
+- DeepStreamSDK 6.2
+- Python 3.8
+- Gst-python
+
+To run the test app:
+  $ python3 deepstream_custom_binding_test.py <h264_elementary_stream>
+
+This document shall describe the sample deepstream-custom-binding-test application.
+
+It is meant for simple demonstration of how to use NvDsUserMeta to attach custom
+data structure to the buffer at upstream element's pad using probe function and extract
+the same custom data structure later at a downstream element's pad, also using another
+probe function.
+
+The elements used in the pipeline are as follows:
+FileSrc -> H264Parse -> NvV4l2Decoder -> NvStreammux -> Queue -> Queue -> FakeSink
+
+First probe function attaches custom structure, which is added to the PyDS bindings, to the NvDsUserMeta, which is added to the frame by PyDS using pyds.nvds_add_user_meta_to_frame()
+This first probe is added at the srcpad of NvStreamMux.
+
+For copying integers and strings from python data structure to PyDS bindings,
+the function pyds.copy_custom_struct() is used.
+
+Second probe function reads the values attached as part of this custom structure after
+the buffer reaches sinkpad of FakeSink. The allocated memory is then freed using pyds.release_custom_struct()
+
+