Video frame capabilities and attributes

[tonerdo]

For more information about how to contribute to this repo, visit this page.

Description

This updates the internal video frame API to allow a video app to specify required capabilities to be applied to the video frame and also read arbitrary attributes on the video frame added by any of the specified capabilities.

Main changes in the PR:

1. Added requiredCapabilities field to VideoFrameConfig.
2. Added attributes field to VideoFrameData.
3. Updated extractVideoFrameAndMetadata to read and populate the attributes field.

Validation

Validation performed:

1. orange web
2. Testing with internal bits to ensure specified capabilities and attributes are successfully applied to video frames

Unit Tests added:

Updated existing videoEffectsEx unit tests.

End-to-end tests added:

Updated existing VideoEffectsExAPI e2e tests.

Additional Requirements

Change file added:

Ensure the change file meets the formatting requirements.

Yes

Next/remaining steps:

List the next or remaining steps in implementing the overall feature in subsequent PRs (or is the feature 100% complete after this?).

• Update Teams client to use requiredCapabilities field
• Update Avatars app to read attributes field

[AE-MS]

?

What is the expectation on the difference between when someone receives undefined for this value vs when they receive a defined but empty map object?

[tonerdo]

attributes will be undefined for hosts that do not (yet) support using the required capabilities specified by the app. Otherwise, it will be empty for apps that don't specify any required capabilities.

[AE-MS]

🕐

[AE-MS]