VPF Tegra support (HW enc acceleration missing) #515

RomanArzumanyan · 2023-08-10T08:06:40Z

Hello world,

This PR adds MVP Tegra support for VPF.

What's new:

PyNvDecoder and PyNvEncoder classes aren't compiled and exported to PyNvCodec module for Tegra. It happens because Tegra doesn't support Video Codec SDK and relies on V4L2 instead.
HW decoding acceleration is supported through PyFfmpegDecoder class.

How to build:
I did it on Tegra board itself, it takes couple minutes

Use FFmpeg build for Tegra supplied by NVIDIA: apt install ffmpeg. It has some limitations like the lack of MP4 containers support. Please be aware of that.
Export TEGRA_BUILD env. variable: export TEGRA_BUILD. It's required because to my best knowledge there's no reliable way to auto-detect Tegra platform. Please correct me if I'm wrong, I'll be happy to get rid of this extra step.
Run usual installation script: python3 -m pip install .

How to run:

To use HW decoding acceleration, use PyFfmpegDecoder and choose appropriate codec:

# VP9 file for example.For H.264 it will be h264_nvv4l2dec corresp.
nvDec = nvc.PyFfmpegDecoder("/path/to/video/i/have/used/vp9.webm, {"c:v": "vp9_nvv4l2dec"})

That's the only difference, rest is the same.
There's more feature-reach Tegra FFmpeg build with decoding and encoding support: https://github.com/jocover/jetson-ffmpeg
But for some reason my build segfaults on Xavier NX board I've used for development.

Future directions:
Feature parity with Video Codec SDK builds.
Fix https://github.com/jocover/jetson-ffmpeg segfaults, then add HW acceleration for encoding.

P. S.
For those who like to build FFmpeg from source, here's how you can substitute apt install ffmpeg step:

# This will download NVIDIA FFmpeg build package sources.
apt source ffmpeg
cd ffmpeg-4.2.7  
mkdir -p build_release_shared

./configure \
  --prefix=$(pwd)/build_release_shared \
  --enable-nvv4l2dec \
  --enable-libv4l2 \
  --disable-doc \
  --disable-static \
  --enable-shared \
  --enable-pic \
  --extra-cflags="-I/usr/src/jetson_multimedia_api/include" \
  --extra-libs="-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils -lv4l2"

make -j6 -s && make install

cd  ./build_release_shared
export LD_LIBRARY_PATH=$(pwd)/lib:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$(pwd)/lib/pkgconfig

gedoensmax

I gave it a quick review @rarzumanyan if you think this does not brake anything etc I am fine merging this. As I have no HW to test this and we are not committed to this I'll make it your decision to have this in master or not.

gedoensmax · 2023-08-15T10:21:14Z

src/PyNvCodec/CMakeLists.txt

+ src/PyBufferUploader.cpp
+ src/PyCudaBufferDownloader.cpp
+ src/PyFFMpegDecoder.cpp
+ src/PyFFMpegDemuxer.cpp
+ src/PyFrameUploader.cpp
+ src/PyNvCodec.cpp
+ src/PySurface.cpp
+ src/PySurfaceConverter.cpp
+ src/PySurfaceDownloader.cpp
+ src/PySurfaceRemaper.cpp
+ src/PySurfaceResizer.cpp


This would be nicer using a CMake variable outside the if as you would have to change 2 locations in the future otherwise.

You're probably right, it can be substituted with list(APPEND ...)
Could you please do it when merging PR?

gedoensmax · 2023-08-15T10:24:26Z

src/TC/src/FfmpegDecoder.cpp

- bool SaveGRAY12LE(AVFrame* pframe) {
- size_t size = frame->width * frame->height * 2;
-
- if (!dec_frame) {
- dec_frame = Buffer::MakeOwnMem(size);
- } else if (size != dec_frame->GetRawMemSize()) {
- delete dec_frame;
- dec_frame = Buffer::MakeOwnMem(size);
- }
-
- auto plane = 0U;
- auto* dst = dec_frame->GetDataAs<uint8_t>();
-
- auto* src = frame->data[plane];
- auto width = frame->width;
- auto height = frame->height;
-
- for (int i = 0; i < height; i++) {
- memcpy(dst, src, 2*width);
- dst += 2*width;
- src += frame->linesize[plane];
- }
-
- return true;
- }
-


Any reason for removing this ?

That probably happened because I created PR, then #502 was merged, and only then I submitted #515
So the answer is no, this is unintentional.

src/TC/src/FfmpegDecoder.cpp

RomanArzumanyan · 2023-08-15T10:41:28Z

As I have no HW to test this and we are not committed to this I'll make it your decision to have this in master or not.

I totally understand, NP.
This was a popular request some time ago (Tegra support) so I hope it will be useful for community.
I'm planning to use it on Tegra in production so at least for the time being it will be tested and supported.

gedoensmax reviewed Aug 15, 2023

View reviewed changes

RomanArzumanyan mentioned this pull request Sep 15, 2023

Does vpf now support jetson devices？ #523

Open

RomanArzumanyan closed this Oct 7, 2023

RomanArzumanyan force-pushed the master branch from fdc8591 to 82b51e7 Compare October 7, 2023 16:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VPF Tegra support (HW enc acceleration missing) #515

VPF Tegra support (HW enc acceleration missing) #515

RomanArzumanyan commented Aug 10, 2023 •

edited

Loading

gedoensmax left a comment

gedoensmax Aug 15, 2023

RomanArzumanyan Aug 15, 2023

gedoensmax Aug 15, 2023

RomanArzumanyan Aug 15, 2023

RomanArzumanyan commented Aug 15, 2023 •

edited

Loading

VPF Tegra support (HW enc acceleration missing) #515

VPF Tegra support (HW enc acceleration missing) #515

Conversation

RomanArzumanyan commented Aug 10, 2023 • edited Loading

gedoensmax left a comment

Choose a reason for hiding this comment

gedoensmax Aug 15, 2023

Choose a reason for hiding this comment

RomanArzumanyan Aug 15, 2023

Choose a reason for hiding this comment

gedoensmax Aug 15, 2023

Choose a reason for hiding this comment

RomanArzumanyan Aug 15, 2023

Choose a reason for hiding this comment

RomanArzumanyan commented Aug 15, 2023 • edited Loading

RomanArzumanyan commented Aug 10, 2023 •

edited

Loading

RomanArzumanyan commented Aug 15, 2023 •

edited

Loading