This application generates real-time facial animations for 3D models based on input sound files.
A short presentation can be seen on my YouTube channel:
The application works in several steps:
- Load speech samples in windows up to 30 ms
- Extract features from the sample windows using linear prediction
- Classify extracted feature blocks to visemes using neural networks
- Animate the 3D model using viseme data