Skip to content

[ECCV 2022] AutoTransition: Learning to Recommend Video Transition Effects

Notifications You must be signed in to change notification settings

acherstyx/AutoTransition

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AutoTransition: Learning to Recommend Video Transition Effects

This is an official implementation of AutoTransition: Learning to Recommend Video Transition Effects.

Transition Dataset

We release the videos with annotated transitions extracted from the video editing template on online video editing platforms. Due to the privacy policy, we only release the link to the videos. The dataset can be downloaded from here: Download Link

Use the following command to download the source video in the dataset:

python3 tools/download_videos.py annotation.json ./template_download

The videos will be downloaded to ./template_download.

Usage

Prepare Data

To speed up the training, we convert videos to JPEG image and extract audio features before training. Run the following commands to finish these steps:

python3 preprocess/convert_video_folder.py ./path/to/template_root
python3 preprocess/extract_audio_features.py ./path/to/template_root path/to/annotation.json --model_path /path/to/audio/model.pth --cuda

The pretrained Harmonic CNN model could be downloaded from this link.

Train & Test

To train transition embeddings:

python3 tools/run_net.py --cfg configs/base/train_transition_embedding.yaml \
  DATASET.TRANSITION_CLASSIFICATION.JSON_ANNOTATION /path/to/annotation.json \
  DATASET.TRANSITION_CLASSIFICATION.TEMPLATE_ROOT /path/to/template_root

The transition embeddings can be found in ./log directory after training.

To train transition recommendation:

python3 tools/run_net.py --cfg configs/base/train_transition_recommendation.yaml \
  MODEL.TRANSITION_TRANSFORMER.EMBEDDING.PRETRAINED_EMBEDDING /path/to/pretrained/transition/embedding.pth \
  DATASET.TRANSITION_DATASET.JSON_ANNOTATION /path/to/annotation.json \
  DATASET.TRANSITION_DATASET.TEMPLATE_ROOT /path/to/template_root

tensorboard --logdir=./log

About

[ECCV 2022] AutoTransition: Learning to Recommend Video Transition Effects

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published