Skip to content

Provides all information and scripts how the CTA Wave Test Content is generated

License

Notifications You must be signed in to change notification settings

cta-wave/Test-Content-Generation

Repository files navigation

Test-Content-Generation

Overview

This repository provides the information and scripts to generate the CTA Wave Test Content.

The run-all.py profiles/csv_file script gathers the data and content from input tables/parameters. Then it sends them for processing. Then it uploads the result.

The processing happens mainly in encode_dash.py. This script is primarily about the usage of GPAC leveraging libavcodec with x264 and x265 to generate the CMAF content along with a DASH manifest. The intent is to keep the size of the post-processing (e.g. manifest manipulation) as small as possible.

Workflow

  • Download mezzanine content from https://dash.akamaized.net/WAVE/Mezzanine/. See section below.
  • Launch scripts:
    • Encode mezzanine content:
      • Encode to conform to CTA Proposed Test content.
      • Encode at least one option of source content according to media profile.
      • NB; the codec value "copy" instructs to bypass the encoding. Useful for proprietary codecs such as DTS or Dolby.
    • Package (markup) the content with an MPD according to the CTA Content Model format.
      • NB: done in Python right now, but could eventually an extension to GPAC to produce this.
    • Encrypt the content in-place using GPAC encryption and manifest-forwarding capabilities.
    • Generate side streams: switching sets, spliced and chunked content, etc.. See sections below.
    • Upload the proposed test content to the CTA-WAVE server using SFTP.
    • Update the Webpage: update database.json.
  • Validate that the content conforms to:

Downloading mezzanine

Sample script for mezzanine v4:

mkdir -p releases/4
cd releases/4
curl http://dash.akamaized.net/WAVE/Mezzanine/releases/4/| sed -n 's/^<IMG SRC=\"\/icons\/generic.gif\" ALT=\"\[FILE\]\"> <A HREF=\"\(.*\)\".*$/\1/p' | grep -v croatia_M1 | grep -v croatia_N1 | grep -v croatia_O1 | xargs -I % wget http://dash.akamaized.net/WAVE/Mezzanine/releases/4/%
cd ..

Encoding to test content

How to generate the content

Main content (clear and encrypted)

Switching Sets (ss1, ss2, etc.)

The generation of current Switching Sets (ss1 for avc, ss2 for hevc/chh1) is done by executing switching_sets/gen_ss1.sh and switching_sets/gen_ss2.sh.

Splicing tests

The generation of current splicing tests is done by executing splice/gen_avc.sh and splice/gen_hevc_chh1.sh.

Chunked tests

The generation of current chunked tests is done by executing chunked/gen.sh cfhd t16 and chunked/gen.sh chh1 t2.

Audio content (XPERI/DTS)

Comment/uncomment the inputs array entries in run-all.py. Then ./run-all.py profiles/dtsc.csv to generate the dtsc content.

Validation

Validation as of today is done manually.

The process of validation includes:


Audio content (AAC / AC-4 / E-AC-3)

The workflow to generate AAC / AC-4 / E-AC-3 audio content is essentially as described here:

  1. Download mezzanine content.
  2. Encode to test content.
  3. Generate DASH segments and manifest.
  4. Patch ISOBMFF.
  5. Patch MPD according to the CTA Content Model format.
  6. Validate content.
  7. Upload to CTA WAVE server and update the Webpage.

Note, steps 1 and 7 are the same as for video content so not repeated.

Prerequisites

  1. Mezzanine content.

  2. Encoding tool:

a) For AC-4/E-AC-3: Dolby Hybrik tool with AC-4 & E-AC-3 support available here: https://davp.dolby.hybrik.com
Note, you need to upload the Pseudo-Noise mezzanine files here first: http://dolby.ibmaspera.com/; sign in using your IBM ID and the login details for the Hybrik online encoder (provided by Dolby). You may be prompted to download a plugin to support upload/download, if so, install the plugin.

b) For AAC: ffmpeg WITH libfdk_aac available here: https://www.gyan.dev/ffmpeg/builds/
Note, the standard FFmpeg build available doesn't include the AAC library (libfdk). As this library is critical, you will need to build the FFmpeg from the source; instructions are available here: FFmpeg: compilation_guide.

  1. Latest version of GPAC available here: https://gpac.io/downloads/gpac-nightly-builds/

2. Encoding

Content and encoding options are documented for AAC & AC-4/E-AC-3 in this Sparse Matrix.

  • AC-4/E-AC-3: use the Dolby Hybrik online encoder tool.

Ensure you have uploaded the Pseudo-Noise mezzanine files as per the pre-requisites. Log in to Hybrik here.

a) AC-4
i. Select Jobs – Create, fill in the form with the following details, then click Submit job.

Template: Encode.wav to AC-4

Job Name: <to confirm when encoding is complete>
Source: <path to PN.wav file>
Destination Path: <folder in which to put encoded content>
Output file (.mp4): <name of encoded file>

Channel Mode: 2.0
Audio Bitrate: 64
Frame rate: 30
I-Frame interval: 60
Language: eng
2nd CM/AD Language Tage: spa
Loudness Regulation Type: EBU R128

ii. Go to Jobs – All jobs to monitor the job.

b) E-AC-3
i. You need to create a JSON file for each audio track you wish to encode, an example is as follows:

{
    "definitions": {
        "job_name": "<insert job name>",
        "source_file": "<insert source file path>/PN01.wav",
        "destination_path": "<insert destination path>",
        "destination_file": "<insert output filename>"
    },
    "name": "{{job_name}}",
    "task_tags": [
        "<insert tags>"
    ],
    "payload": {
        "elements": [
            {
                "uid": "source_file_a",
                "kind": "source",
                "payload": {
                    "kind": "asset_url",
                    "payload": {
                        "storage_provider": "s3",
                        "url": "{{source_file}}",
                        "access": {
                            "credentials_key": "<insert key>"
                        }
                    }
                }
            },
            {
                "uid": "transcode_task",
                "kind": "transcode",
                "task": {
                    "retry_method": "fail"
                },
                "payload": {
                    "location": {
                        "storage_provider": "s3",
                        "path": "{{destination_path}}",
                        "access": {
                            "credentials_key": "<insert key>"
                        }
                    },
                    "targets": [
                        {
                            "file_pattern": "{{destination_file}}",
                            "container": {
                                "kind": "mp4"
                            },
                            "audio": [
                                {
                                    "codec": "eac3",
                                    "channels": 2,
                                    "sample_rate": 48000,
                                    "bitrate_kb": 128,
                                    "language": "eng"
                                }
                            ]
                        }
                    ]
                }
            }
        ],
        "connections": [
            {
                "from": [
                    {
                        "element": "source_file_a"
                    }
                ],
                "to": {
                    "success": [
                        {
                            "element": "transcode_task"
                        }
                    ]
                }
            }
        ]
    }
}

ii. Go to Jobs – All jobs, then click Submit Job JSON to upload file. Any errors with the JSON file will be given at upload.

  1. Download content i. File are downloaded from: http://dolby.ibmaspera.com/ (note, there may be a slight delay in the file appearing). ii. Right click a file and select Download to.

b) AAC: use ffmpeg with the following command (depending on the required codec):

a) AAC-LC:

ffmpeg -i {source} -c:a libfdk_aac aac_lc -ar {Sample Rate} -b:a {Bitrate} {channel_config} -t {Duration} -use_editlist {elst_present} {output1}.mp4

b) HE-AAC or HE-AAC-V2:

ffmpeg -i {source} -c:a libfdk_aac -profile:a {codec} -ar {Sample Rate} -frag_duration 1963000 -flags2 local_header -latm 1 -header_period 44 -signaling 1 -movflags empty_moov -movflags delay_moov -b:a {Bitrate} {channel_config} -t {Duration} -use_editlist {elst_present} {output1}.mp4

c) Encrypted:

If the content is to be encrypted, it should be done before generating DASH segments; MP4box (part of gpac) supports encrypting media.

3. Generate DASH segments and manifest

To DASH the content gpac is used with the following command:

gpac -i {output1}.mp4:FID=A1 -o {output2}.mpd:profile=live:muxtype=mp4:segdur={segmentduration}:cmaf=cmf2:stl:tpl:template="1/$Time$":SID=A1'

4. Patching ISOBMFF

This is required to fix metadata stored in some ISOBMFF boxes that make up the DASH segments, or fixing the structure/format.

An example of what needs to be changed is: styp compatibility brands set to ["msdh", "msix", "cmfs", "cmff", "cmf2", "cmfc", "cmf2"]

5. Patching MPD to CTA Content Model

The required changes to the MPD are defined in the CTA WAVE Device Playback Capabilities Specification CTA-5003-A (current version: CTA-5003-A, Published September 2023), section “5.3.4.2 Content Model Format for Single Media Profile”.

An example of a change to the MPD is adding the CTA copyright notice.

6. Validation

Content is validated to ensure conformity with:

About AAC / AC-4 / E-AC-3 Audio

The initial set of audio test vectors as defined in the Sparse Matrix were created by Resillion on behalf of Fraunhofer IIS for AAC, and on behalf of Dolby for AC-4/E-AC-3. Any future requests for content should be addressed to Dolby or Fraunhofer IIS, respectively.

About

Provides all information and scripts how the CTA Wave Test Content is generated

Resources

License

Stars

Watchers

Forks

Packages

No packages published