Skip to content

πŸ“Ί πŸ“° πŸ§‘β€πŸ’Ό Toolkit for skeleton & face analysis of talking heads (e.g. news) videos πŸ§‘β€πŸ’Ό πŸ“° πŸ“Ί

License

Notifications You must be signed in to change notification settings

frankier/skelshop

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

SkelShop

Docker/Singularity containers | Documentation

SkelShop is a toolkit usable either as a suite of command line tools or a Python library aimed at offline analysis of the ``talking heads'' genre of videos, typically created in a television studio. This genre includes news and current affairs programmes, variety shows and some documentaries. The main attributes of this type of material is:

  • Multiple fixed place cameras in a studio with shot changes to different cameras and non-studio material.
  • Mainly upper bodies are visible.
  • Frontal shots are the most frequent type.
  • Faces are usually visible and most often frontal.
  • Occlusion is as often by an OSD (On Screen Display) than by some other object in the scene.

Getting started

See the documentation.

Features

  • Dump OpenPose skeletons to a fast-to-read HDF5 file format
  • A processing pipeline starting with shot segmentation
  • Apply black box person tracking on OpenPose skeletons
  • Draw skeletons over video and...
    • View the result in real time
    • Output the result to another video
  • Convert from some existing JSON based dump formats
  • Embed faces using dlib
    • Using OpenPose's face detection and keypoint estimation
    • Or using dlib's own face detection/keypoint estimation
    • Select best frames for best quality embeddings
  • Dump across heterogeneous HPC environments including GPU and CPU nodes.
  • Identify faces based on embeddings
    • Identify by direct comparison with references
    • Or cluster faces with no reference (also useful re-identitication)
    • Identify faces based on clusters, either with a reference or interactively
    • Quickly build up libraries of references using data from WikiData/WikiMedia commons

Here's a screenshot of the playsticks command:

Screenshot of the playsticks command

Find out more about what SkelShop can do in the documentation.

Contributions & Questions

Contributions are welcome! Feel free to use GitHub discussions to ask any questions. See also the contributing section of the documentation.

Acknowledgments

Thanks to the authors of all the useful libraries I have used.

Some of the black box tracking code is based on this repository.

Some code to do with clustering and usage of FAISS is based on this repository.

Icon by Adrien Coquet, FR from the Noun Project used under CC-BY.

About

πŸ“Ί πŸ“° πŸ§‘β€πŸ’Ό Toolkit for skeleton & face analysis of talking heads (e.g. news) videos πŸ§‘β€πŸ’Ό πŸ“° πŸ“Ί

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Languages