This repository stores code from my experiences with the Posenet model, which is used to estimate human poses from within a web browser in real-time.
In this repo, you'll find the codes for Pose Estimation and Image Classification. I used P5.js and ML5.js libraries and followed The Coding Train tutorial.
There is a set of computer vision technicques that detect human figures in images or videos and estimates where key body joints are; it's called pose estimation. PoseNet is a Machine Learning model that is able to estimate real-time human poses, by using ml5.js library we can do it directly from the browser using a webcam capture.
You can follow the steps below and try it out on your own computer.
Just so we don't need to install anything on our computer, let's use cdn links to access p5.js and ml5.js libraries:
<!DOCTYPE HTML>
<html>
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>PoseNet experiments</title>
<script src="https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js"></script>
<script src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js"></script>
<script language="javascript" type="text/javascript" src="YOURSKETCH.js"></script>
</head>
<body>
</body>
</html>We can also use the P5.js web editor or install p5.js and ml5.js using our package manager
In this step, we're gonna create a sketch to capture the video from the webcam, load the PoseNet model and use it to identify your key body joints and draw lines connecting them - forming a skeleton.
Here you can find the source code for a full body pose estimation.
Here you can find the source code for drawing a red circle (clown nose) on your face.
This step refers to single-person pose estimation, that is, when there's only person centered in the input image/video. If there are more than one person, the multi-pose estimation algorithm should be used instead.
For Pose Classification, we need to follow these steps:
- Setup a neural network;
- Collect data and feed it to the neural network;
- Train and save the model;
- Classify new poses.
The pose-classifier directory has 3 files:
- setup-collect for the first two steps described above;
- train-model for training the model with the data we just collected;
- classifier to make the new poses classification.
It also has a folder 'model' that stores three files generated by the model training step.