Web-avatarify for image animations

This is the code base for this website and its backend. This aims to bring technology closer to everyone, just by a couple of clicks away, democratizing it to anyone. It is based on avatarify which uses the incredible work of a recent paper. Thanks for the authors for releasing the code and the trained model.

In addition it allows you to record and produce a video also with your input audio.

Jon Snow Example with audio

Updates

• It will be running until June 26 where my Azure Free credits for running the GPU backend are over :D
• June 8 2020 web page public released

Architecture

High level info

• The Web App was made using React (actually I learned it for building this so don't except clean javascript code :D).
• Everything is containerized so it should run everywhere.
• There is a version for running it in CPU (Dockerfile) and a version for GPU (Dockerfile-gpu) -because at the beginning I thought people could wait a couple of minutes whilst processing the video :D, I was wrong, it takes too much on CPU-
• Because the WebApp needs access to the browser's microphone and camera, it needs to be served over HTTPS. I used firebase for hosting the app because its easytouse features and automatic SSL handling, and also custom name routing.
• I purchased the domain on https://freenom.com because they have 1 year free .ml domains.
• The backend that runs the model (final version with GPU) were deployed in azure in a single Virtual Machine (because this is just a demo and more machines will cost considerably more).
• One tricky implementation detail is that because the front end is served over HTTP, the backend also needs to be served over HTTPS. This will complicate things a lot, so at the end the fast solution (I know not the most elegant one) was to use a relay server with HTTPS using Google Cloud Run (manages automatically the HTTPS part) that just redirects the petitions to the virtual machine.

TODO: Image with the actual architecture :P

Requirements

• WebApp tested on mobile and desktop machines with Chrome and Mozilla browsers.
• React to test front end locally
• Firebase command for deploying front end
• Gcloud command for deploying part of backend
• Python 3.7 and requirements.txt for the backend
• A secrets.sh file with variables API_TOKEN and SERVER_URL. The first one is the token for accessing the backend and the last one the url of the backend. Also add this at the end export REACT_APP_API_TOKEN=$API_TOKEN for the front end to get the variables.

Running the web-app

This is straightforward like any react app. Just make sure to source the secrets.sh file before.

source secrets.sh
cd web-app/
npm install
npm start

For deploying:

npm run build
firebase deploy

Running the backend

Locally

bash scripts/run-local.sh

Locally with Docker

bash scripts/run-docker.sh

or for GPU

bash scripts/run-docker.sh --gpu

Deploy to cloud run

This will deploy the CPU version (reaaaaally slow)

bash scripts/deploy.sh