FFmpeg Integration

Currently there is no official support for VVC in FFmpeg. But community patches have been submitted, so it is possible to apply them manually.

The submitted patch series is called 'Add support for H266/VVC' and adds support for VVdeC and VVenC to FFmpeg.

• To build FFmpeg the general compilation guide is needed:
  https://trac.ffmpeg.org/wiki/CompilationGuide

• To patch FFmpeg for VVenC and VVdeC support follow the guide depending on your operating system.

Linux

0. Build and install VVenC and VVdeC where it can be found by pkg-config

• Build VVenC and VVdeC shared and install into a system path, e.g.:
  sudo make install install-prefix=/usr/local

1. Clone FFmpeg and to into the downloaded folder

• git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
• cd ffmpeg

2. Checkout latest working FFmpeg version

• git checkout 2532e832d2

3. Download the latest VVC patchset from ffmpeg patchwork

• wget -O Add-support-for-H266-VVC.patch https://patchwork.ffmpeg.org/series/9992/mbox/

4. Patch FFmpeg with the downloaded patch file for VVC support

• git apply --check Add-support-for-H266-VVC.patch

• git apply Add-support-for-H266-VVC.patch

  Attention: In case git apply --check fails caused by the libavcodec/version.h like:
  error: patch failed: libavcodec/version.h:29
  you can force to apply the patch and ignoring the version number by using:
  git apply Add-support-for-H266-VVC.patch --exclude=libavcodec/version.h

  If other issues occur, please use latest working revision 2532e832d2
  clean the master, checkout and apply patch:
  git reset --hard
git clean -df
git checkout 2532e832d2
git apply Add-support-for-H266-VVC.patch

5. Configure and build FFmpeg

• ./configure --enable-pthreads --enable-pic --enable-shared --enable-rpath --arch=amd64 --enable-demuxer=dash --enable-libxml2 --enable-libvvdec --enable-libvvenc
  • Add more options to your configure command to enable all needed tools and codecs
• make (use make -j to compile multi-threaded)

6. Install ffmpeg (optional step)

• sudo make install

7. Check if FFmpeg supports VVC
   ffmpeg -hide_banner -codecs | grep vvc
DEV.L. vvc H.266 / VVC (Versatile Video Coding) (decoders: libvvdec ) (encoders: libvvenc )

• if libvvdec and libvvenc are not listed, something went wrong

In case you want to revert all changes the master can be cleaned up to origin/master by using:
git reset --hard
git clean -df

Using latest FFmpeg release 6.1 ("Heaviside")

To apply the latest patchset libavcodec/version.h must be ignored.
Checkout ffmpeg n6.1, download latest patchset and apply:
git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg
git checkout release/6.1
wget -O Add-support-for-H266-VVC.patch https://patchwork.ffmpeg.org/series/9992/mbox/
git apply Add-support-for-H266-VVC.patch --exclude=libavcodec/version.h

MacOS

In general you can follow the guidelines for Linux when build on MacOS.

VVenC and VVdeC is also available in Homebrew.
If you want to skip the build process it can be installed by calling:
brew install vvdec vvenc

When building on an ARM platform please follow this guide:

ARM devices (M1, M2)

When running on an ARM device you have to build and install VVenC, VVdeC and FFmpeg for ARM To build VVenC and VVdeC on ARM ninja build must be used to build, as with XCode generated libraries are not working.
It can be installed by using Homebrew: brew install ninja

1. Build and install VVdeC:
   sudo make g=ninja install install-prefix=/usr/local
   Alternatively install via Homebrew: brew install vvdec
2. Build and install VVenC:
   sudo make g=ninja install install-prefix=/usr/local
   Alternatively install via Homebrew: brew install vvenc
3. configure and build FFmpeg (after patch has been applied):
   ./configure --enable-pthreads --enable-pic --enable-shared --enable-rpath --arch=arm64 --enable-demuxer=dash --enable-libxml2 --enable-libvvdec --enable-libvvenc

Windows

As building FFmpeg for Windows can be tricky, we recommend the Media Autobuild Suite.
VVenC and VVdeC are already part of the suite and must be enabled during the script setup to get VVC support in FFmpeg.

Follow the guide given in the Media Autobuild Suite to build FFmpeg.

1. Clone the media-autobuild_suite

• git clone https://github.com/m-ab-s/media-autobuild_suite
• cd media-autobuild_suite

2. Patch media-autobuild_suite to enable ffmpeg with vvc support

• Open the file media-autobuild_suite/build/media-suite_compile.sh in your favorite Editor
• Search for line lib/cmake/vvenc/vvencConfig.cmake):

if [[ $bits = 64bit && $vvenc = y ]] &&
     do_vcs "$SOURCE_REPO_LIBVVENC"; then
 # vvenc implementation
fi

• change the line fi to following code:

else
    pc_exists libvvenc || do_removeOption "--enable-libvvenc"
fi

• Search for line lib/cmake/vvdec/vvdecConfig.cmake):

if [[ $bits = 64bit && $vvdec= y ]] &&
     do_vcs "$SOURCE_REPO_LIBVVDEC"; then
 # vvdecimplementation
fi

• change the line fi to following code:

else
    pc_exists libvvdec || do_removeOption "--enable-libvvdec"
fi

• Search for line do_changeFFmpegConfig:

do_changeFFmpegConfig "$license"

• Go into next line and insert the following code to apply the patchset:

if [[ $bits = 64bit ]] && enabled_any libvvdec libvvenc; then
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231103095720.32426-2-thomas.ff@spin-digital.com/mbox/" am  ||
          do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231103095720.32426-3-thomas.ff@spin-digital.com/mbox/" am  ||
          do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231103095720.32426-4-thomas.ff@spin-digital.com/mbox/" am  ||
          do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231103095720.32426-6-thomas.ff@spin-digital.com/mbox/" am  ||
          do_removeOptions "--enable-libvvdec --enable-libvvenc"
fi

• Save and close the file media-autobuild_suite/build/media-suite_compile.sh
• Open the file media-autobuild_suite/media-autobuild_suite.bat in your favorite Editor
• Search for the line:
  libdav1d libaom --disable-debug libfdk-aac
• Change the line to:
  libdav1d libaom --disable-debug libfdk-aac libvvdec libvvenc
• Save and close the file media-autobuild_suite/media-autobuild_suite.bat

3. Open the Windows PowerShell and run:
   media-autobuild_suite.bat

4. Follow the instructions given by the script and enable VVenC and VVdeC when asked

Using latest FFmpeg release 6.0

Using the latest ffmpeg release n6.0 which is compliant with the patchset v6.
In General follow the same guide as given above.
To set a specific ffmpeg version or tag you have to adapt the file media-autobuild_suite.ini:

• Open the file media-autobuild_suite/build/media-autobuild_suite.ini in your favorite Editor
• Search for line ffmpegPath and change the uri to the following line:
  ffmpegPath=https://git.ffmpeg.org/ffmpeg.git#tag=n6.0
• Save and close the file media-autobuild_suite/build/media-autobuild_suite.ini

To apply the patchset v6 please add following code into media-autobuild_suite/build/media-suite_compile.sh under the line:

do_changeFFmpegConfig "$license"

if [[ $bits = 64bit ]] && enabled_any libvvdec libvvenc; then
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-2-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-3-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-4-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-5-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-6-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-7-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-8-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-9-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-10-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230210174106.44514-11-thomas.ff@spin-digital.com/mbox/" am  ||
		  do_removeOptions "--enable-libvvdec --enable-libvvenc"
fi

Troubleshooting: Media Autobuild Suite

FFmpeg may fail to build as the Media Autobuild Suite is kind of a moving target.
That means FFmpeg is depending on a bunch of core applications and codecs which are build from the current master branch per default. As a depending repository may fail to build or is not compatible with the ffmpeg version this section will show solutions or workarounds on known issues.

libjxl

• error during ffmpeg configure step: ERROR: libjxl >= 0.7.0 not found using pkg-config
• workaround: disable libjxl in file build/media-autobuild_suite.ini
jpegxl=2 and in file build/ffmpeg_options.txt by comment line
  #--enable-libjxl

libavif

• workaround: disable libavif in file build/media-autobuild_suite.ini
libavif=2

texinfo 7.1 not compatible with current ffmpeg head

• error during ffmpeg build step: /build/ffmpeg-git/doc/t2h.pm: Undefined subroutine &Texinfo::Config::set_from_init_file ...
• workaround: disable html generation in file build/ffmpeg_options.txt by adding
  --disable-doc

libplacebo

• workaround: disable in file build/ffmpeg_options.txt by comment line #--enable-libplacebo or setting
  --disable-libplacebo

set specific revisions

If a dependent library has changed the interface and is not compatible with ffmpeg anymore a particular revision can be set in the file build/media-suite_deps.sh, e.g. for mpv player
SOURCE_REPO_MPV=https://github.com/mpv-player/mpv.git#tag=v0.36.0
The particular FFmpeg revision can also be set in the file build/media-autobuild_suite.ini, e.g.
ffmpegPath=https://git.ffmpeg.org/ffmpeg.git#2532e832d2

Usage

Check for valid FFmpeg installation

ffmpeg -hide_banner -codecs | grep vvc
DEV.L. vvc H.266 / VVC (Versatile Video Coding) (decoders: libvvdec ) (encoders: libvvenc )

Available options

Run FFmpeg help to see available options for encoder and decoder:
ffmpeg -h encoder=libvvenc

Encoder libvvenc [H.266 / VVC Encoder VVenC]:
General capabilities: delay threads
Threading capabilities: other
Supported pixel formats: yuv420p10le

libvvenc-vvc encoder AVOptions:
-preset <int> E..V....... set encoding preset(0: faster - 4: slower (from 0 to 4) (default medium)
faster 0 E..V....... 0
fast 1 E..V....... 1
medium 2 E..V....... 2
slow 3 E..V....... 3
slower 4 E..V....... 4
-qp <int> E..V....... set quantization (from 0 to 63) (default 32)
-period <int> E..V....... set (intra) refresh period in seconds (from 1 to INT_MAX) (default 1)
-subjopt <boolean> E..V....... set subjective (perceptually motivated) optimization (default true)
-bitdepth8 <boolean> E..V....... set 8bit coding mode (default false)
-vvenc-params <dictionary> E..V....... set the vvenc configuration using a :-separated list of key=value parameters
-levelidc <int> E..V....... vvc level_idc (from 0 to 105) (default 0)
0 0 E..V....... auto
1 16 E..V....... 1
2 32 E..V....... 2
2.1 35 E..V....... 2.1
3 48 E..V....... 3
3.1 51 E..V....... 3.1
4 64 E..V....... 4
4.1 67 E..V....... 4.1
5 80 E..V....... 5
5.1 83 E..V....... 5.1
5.2 86 E..V....... 5.2
6 96 E..V....... 6
6.1 99 E..V....... 6.1
6.2 102 E..V....... 6.2
6.3 105 E..V....... 6.3
-tier <int> E..V....... set vvc tier (from 0 to 1) (default main)
main 0 E..V....... main
high 1 E..V....... high

The decoder doen´t have any additional options. Capabilities can be shown by:
ffmpeg -h decoder=libvvdec

Decoder libvvdec [H.266 / VVC Decoder VVdeC]:
General capabilities: delay threads
Threading capabilities: other
Supported pixel formats: gray gray10le yuv420p yuv422p yuv444p yuv420p10le yuv422p10le yuv444p10le

Usage

Basic

• Encode a RAW video file with VVenC into mp4:
  ffmpeg -f rawvideo -vcodec rawvideo -s 1920x1080 -framerate 25 -pix_fmt yuv420p -i file_1080p_25Hz_420_8bit.yuv -an -vcodec vvc output.mp4

• Encode with VVenC by using a preset and bitrate:
  ffmpeg -i <input> -c:v vvc -b:v 2600k -preset faster <output>

available presets: faster,fast,medium,slow,slower

• Set thread count (default: auto detection)
  ffmpeg -i <input> -an -vcodec vvc -threads 6 <output>

Encoder specific parameter (vvenc-params)

• Set any available VVenC parameter via -vvenc-params (separated by :) e.g.
  ffmpeg -i <input> -an -vcodec vvc -vvenc-params bitrate=1M:passes=2:pass=1 <output>

• Combines command with all available VVenC options:
  ffmpeg -i <input> -an -vcodec vvc -b:v 2M -subjopt 0 -period 2 -preset medium -vvenc-params passes=2:pass=1 <output>

Special options via vvenc-params

• using low decoder energy preset (preset must be set to medium)
  ffmpeg -i <input> -an -vcodec vvc -preset medium -vvenc-params "NumRefPics=2:DeblockLastTLayers=1:MaxMTTDepth=332222:MaxMTTDepthI=3:Affine=3:ALFSpeed=1:BCW=2:BIO=0:DMVR=0:ISP=0:LFNST=0:LMCSEnable=0:MIP=0:FastMIP=0:SAO=2:SbTMVP=0:FastMrg=2" <output>

Rate control (VBR mode)

• Set target bitrate (VBR mode) (default: 200kbit/s)
  ffmpeg -i <input> -an -vcodec vvc -b:v 2000k <output>
ffmpeg -i <input> -an -vcodec vvc -b:v 2M <output>

• Set 2-pass encoding (default: single pass)

1. run 1st pass:
   ffmpeg -i <input> -an -vcodec vvc -vvenc-params passes=2:pass=1:rcstatsfile=stats.json -b:v 2M -f null /dev/null
2. run 2nd pass:
   ffmpeg -i <input> -acodec copy -vcodec vvc -vvenc-params passes=2:pass=2:rcstatsfile=stats.json -b:v 2M <output>

combined call:

ffmpeg -i <input> -an          -vcodec vvc -vvenc-params passes=2:pass=1:rcstatsfile=stats.json -b:v 2M -f null /dev/null &&  
ffmpeg -i <input> -acodec copy -vcodec vvc -vvenc-params passes=2:pass=2:rcstatsfile=stats.json -b:v 2M <output>  

• rcstatsfile defines a json files where the statistics of the 1st pass are written to.
• If not given a default statsfile 'vvenc-rcstats.json' is used.
• Be aware to use unique statfiles when running several ffmpeg instances!
• If they are not unique all instances writes into the same file which leads to encoder failures
• As first pass does not output anything, you can set a filename or -f null /dev/null (windows: -f null NUL)

• Set fix qp mode (default: off)
  ffmpeg -i <input> -an -vcodec vvc -b:v 0 -qp 25 <output>

PSNR based encoding

The parameter -subjopt for better perceptual quality is enabled per default.
That means the encoder will perform for better subjective quality and better MS-SSIM results.
When this setting is disabled the encoder performs for better PSRN and VMAF results.

• Disable perceptual optimization for better PSNR and VMAF ( default: enabled )
  ffmpeg -i <input> -an -vcodec vvc -subjopt 0 <output>

Intra Period (also known as GOP or keyframe interval) and decoding refresh types

• Set intra period (keyframe interval) to 2 seconds ( default: 1sec )
  ffmpeg -i <input> -an -vcodec vvc -period 2 <output>

• Set intra period (keyframe interval) in frames to 64
  ffmpeg -i <input> -an -vcodec vvc -vvenc-params intraperiod=64 <output>

• Set decoding refreshtype ( idr,cra,idr2,cra_cre, default: cra )
  ffmpeg -i <input> -an -vcodec vvc -vvenc-params "decodingrefreshtype=idr" <output>

idr : closed GOP
cra : open GOP
cra_cre : open GOP for streaming

8-bit coding

Be aware of that this feature is only working for VVenC version >= v1.9.1.
Please be also aware of, the input content is always converted to yuv420p10le as the VVenC library only supports 10-bit input.
ffmpeg -i <input> -an -vcodec vvc -bitdepth8 1 <output>

This feature can also be used via -vvenc-params:
ffmpeg -i <input> -an -vcodec vvc -vvenc-params "InternalBitDepth=8" <output>

HDR and SDR signalization for RAW input

• signalization is only needed when input does not contain specific information
• signalizing SDR BT.709: -vvenc-params Sdr=sdr ( BT.2020 -vvenc-params Sdr=sdr_2020)
• signalizing HDR10/PQ : -vvenc-params Hdr=pq ( BT.2020 -vvenc-params Hdr=pq_2020)
• signalizing HLG : -vvenc-params Hdr=hlg (BT.2020: -vvenc-params Hdr=hlg_2020)

Special Encoding options

• Tile encoding for faster multithreading when using more than 8 threads
  ffmpeg -i <input> -an -vcodec vvc -threads 16 -vvenc-params wavefrontsynchro=1:tiles=2x2 <output>

Transcoding

• Transcode Transport stream file with VVC into ISOBMFF file format:
  ffmpeg -i input.ts -an -vcodec vvc output.mp4

• Extract RAW ES from mp4:
  ffmpeg -i input.mp4 -an -vcodec copy -bsf h266_mp4toannexb output.266

Playback

ffplay is part of ffmpeg and can be used to playback VVC streams. Following file formats are supported and can be played with ffplay:

• RAW elementary streams (.266, .vvc)
• ISO base media file format (.mp4, .mpd)
• MPEG transport streams (.ts)

general usage of ffplay

ffplay input

To playback a specific video track the option -vst v:id can be used. e.g.:
ffplay  dashfile.mpd -vst v:2
To toggle between different tracks during playback use v

Test streams

A variety of transport streams and dash streams according to the DVB specification can be found at:
https://dvb.org/specifications/verification-validation/vvc-test-content/

All test bitstreams from JVET can be found at: https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVC/draft_conformance/draft6/

Disclaimer

This installation guide will only work as long as the patch series can be merged with the FFmpeg master.
The used patchset is an community submitted patch and not merged into the master nor checked by the developer team.
This guide was tested with FFmpeg commit 9413bdc381 and patchset v7.