Skip to content

deeppavlov/MultimodalERC

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Multimodal model with concatenation fusion

python3 multimodal_concat/train.py -n <wandb project name> --lr <learning rate> --batch_size <batch_size> --seed <random seed>

Notes:

  • Each modality features are represented by last hidden state of a modality-specific model
  • Text features are taken from [CLS] token
  • Video features are pooled from 16 frames into 1
  • Audio features are WIP
  • The model consists of two linear layers for now

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 81.5%
  • Python 18.5%