Skip to content

IntelligentSensingAndRehabilitation/GaitTransformer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is code to run the Gait Transformer described in:

Installation

pip install -e .

For further information, refer here.

Usage

See notebooks/demo.ipynb for a demo of the model. See docs for more examples.

To run this on your own data you will need to produce similarly formatted 3D keypoint sequences as this is trained on. This is used with outputs from PosePipe [http://github.com/peabody124/PosePipeline]. The weights for the demo use the MMPose option for producing the 2D keypoints and GastNet for producing the 3D keypoints.

The Gait Transformer with these weights does not work robustly far from the frontal view. Anecodtally, it appears to still be fairly reliable for gait timing but the estimates of velocity become quite biased. I suspect this is due to viewpoint sensitive biases in the 2D->3D lifting step.

Note that the core gait transformer algorithm uses TensorFlow and the Kalman Filter uses Jax. You will need to ensure they both allow each other memory or perform this as separate steps.

Data to reproduce training this model cannot be made available due to privacy protection.

TODO:

  • create more standalone utility functions for some of the functions in gait_decoder_testing
  • retrain using the outputs from MeTRAbs-ACAE
  • ideally have a separate frontend for processing videos with the recommended PosePipe steps to make it easier to test

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published