-
Notifications
You must be signed in to change notification settings - Fork 134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A possibility to track multi-person in the same scene #54
Comments
The weird behavior you are refering to arises from the 2D joint heatmap detection If you get multiple persons in a scene the algorithm will try to "connect" parts of the bodies of different persons ( the parts with the highest score ) resulting in incorrect results.. In the older version of MocapNET ( MNET1 ) there used to be a mode ( https://github.com/FORTH-ModelBasedTracker/MocapNET/blob/mnet1/src/MocapNET1/MocapNETLiveWebcamDemo/mocapNETLiveDemo.cpp#L838 )
If you think you will find this useful then I could reinstate it.. That being said the second thing one can do is use OpenPose with the -number_people_max 1 flag, this way OpenPose will just pick one skeleton and solve the issue. OpenPose uses PAFs that allow joints to be connected on the same person, and has provisions to correctly seperate persons in a scene A proper solution for the live webcam demo would be to incorporate a neural network like Darknet/YOLO ( https://github.com/AlexeyAB/darknet ) run this first on the incoming OpenCV frame, retrieve the persons on the image ( as seen here https://www.youtube.com/watch?v=saDipJR14Lc#t=23m ) and then run the MocapNET pipeline on each of the retrieved rectangles .. This will work, it will also degrade framerate linearly with more persons present in the scene ( since the Neural Network will have to be executed once for each one of them ), then you will also have the additional problem of person reidentification so that you have multiple BVH file outputs and keep track of which skeleton belongs to which BVH file and update them correctly .. So that being said adding all this complexity on the project is overkill and it doesnt have a lot of novelty or research interest so that is why it has been skipped..! I think at this point the best thing to be done is masking parts of the scene you dont want in an attempt to workaround, ( or just use OpenPose as the 2D engine ) Hope I did a good job explaining the issue, Ammar |
Hello,
The code works amazingly for shuffle.webm and other single person stimuli, but works very strangely when I put the multi-person video.
Is there any way to expand MocapNET with multi-person one like
https://paperswithcode.com/task/multi-person-pose-estimation?
The text was updated successfully, but these errors were encountered: