Unsupervised-Learning-of-Deep-Feature-Representation-for-Clustering-Egocentric-Actions

This is the code for our paper "Unsupervised Learning of Deep Feature Representation for Clustering Egocentric Actions, IJCAI 2017".

The folder contains following files.

1. train_2D_mult_auto_ver4.py : Train multilayer 2D convolutional autoencoders to extract frame level features.

2. test_2D_conv.py : Extract spatial features from the video

3. lstm_2D_conv_ver4.py : Train lstm autoencoder to learn splice level representations

4. test_lstm.py : Extract splice level representations

5. opti.py : Extract dense optical flow and save it for each splice

6. raw_frame.py : Extract raw frames and save them for each splice

7. ind2vec.m confusion.m : Helper functions

8. cluster.m : Clusters the features extracted from the LSTM

9. make_gtea.m : Sample code to generate GT in required format (from GTEA dataset in this case)

10. match_greedy2.m : Code for greedy matching of generated labels and GT

11. readNPY.m, readNPYheader.m : Helper functions to read .npy files in matlab

12. metadata_S2.mat : Contains information regarding video length etc.

13. gteagroundtruth : Contains sample files to form GT for GTEA dataset

14. complete.sh : Runs entire feature extraction pipeline (from optical flow extraction to LSTM features)

• In order to run these codes you will need to extract dense optical flow and raw frames from the video (use code opti.py and raw_frames.py).
• All videos were converted to 15 fps and splices were formed at 2 sec for short term actions and 30 sec for long term actions.