A Human Pose Skeleton represents the orientation of a person in a graphical format. Essentially, it is a set of coordinates that can be connected to describe the pose of the person. Each co-ordinate in the skeleton is known as a part (or a joint, or a keypoint). A valid connection between two parts is known as a pair (or a limb).
- Extraction of pose coordinates from dance videos using openpose human pose estimation.
- Training LSTM network on extracted coordinates using video as input and coordinates as output.
- Display output videos by joining predicted coordinates to generate dancing human stick figures.
- You will first have to download the repository and then extract the contents into a folder.
- Make sure you have the correct version of Python installed on your machine. This code runs on Python 3.6 above.
- Now, install the required libraries.
- Now go to src folder and run extract_data.py to download videos and audios to data folder. You can add youtube videos links to "video_links.txt" file for downloading.
- Download pretrained weights for pose estimation from here. Download pose_iter_440000.caffemodel and save it in "models" folder.
- Run demo.py to train LSTM and display predicted dance video.
python main.py --video data/video.mp4 --audio data/audio.wav --background data/bg1.jpg
- Pose estimation using openpose on image
- Pose estimation using openpose on video
NOte : For more information about implementation details, please go through AI Dance based on Human Pose Estimation