- Deep neural networks for classification of multispectral images
- Niklas Lindorfer
- 2265846l
- Professor Roderick Murray-Smith
- This file contains the time log for your project. It will be submitted along with your final dissertation.
- YOU MUST KEEP THIS UP TO DATE AND UNDER VERSION CONTROL.
- This timelog should be filled out honestly, regularly (daily) and accurately. It is for your benefit.
- Follow the structure provided, grouping time by weeks. Quantise time to the half hour.
- 0.5 hour Set up project repository on GitHub
- 1.5 hours Downloaded LaTeX template and set up CD script on GitHub
- 1.0 hour Set up timelog, plan, milestones, etc.
- 3.0 hours Reading and research (100 page ML book)
- 0.5 hours Updated project plan
- 1.0 hour Reading
- 0.5 hours 1st supervisor meeting
- 3.5 hours Reading
- 0.5 hours 2nd supervisor meeting
- 1.0 hour Reading
- 1.0 hour Keras tests
- 2.0 hours Keras tests (image colorization)
- 2.5 hours Reading
- 1.0 hour Reading
- 1.5 hours Keras tests (image colorization)
- 2 hours Reading
- 0.5 hours 3rd supervisor meeting
- 1.0 hour Requirement analysis
- 4.0 hours Reproduced results of a similar previous project
- 3.5 hours Investigation into object detection strategies
- 0.5 hours 4th supervisor meeting
- 3.0 hours Exploration of YOLO object detection model
- 2.0 hours Prepared extraction of thermal and RGB from FLIR sensor export
- 2.0 hours Initialized mobile app and set up repository
- 1.0 hours Got FLIR support to activate my developer account
- 2.0 hours Further work on YOLO model test
- 5.0 hours Familiarization with Flir One SDK
- 0.5 hours 5th supervisor meeting
- 3.0 hours Work on YOLO activation function
- 3.0 hours Classification task with Pytorch
- 3.0 hours Started investigating FLIR One calibration
- 1.0 hour Investigaged FLIR One calibration
- 1.0 hour Dissertation download link
- 3.0 hours Investigated FLIR One calibration
- 0.5 hours 6th supervisor meeting
- 2.0 hours Worked on Android app
- 3.0 hours Worked on Android app
- 2.5 hours Detection research
- 1.5 hour Investigated FLIR One calibration depending on rotation
- 1.5 hours Hands classification
- 0.5 hours 7th supervisor meeting
- 4.0 hours Worked on Android app
- 6.0 hours NN in tensorflow
- 0.5 hours 8th supervisor meeting
- 6.0 hours Refactored and improved Android app. Tensorflow integration should be possible now.
- 1.5 hours Took test dataset and created model to test on mobile
- 7.5 hours Started adding Tensorflow Lite to mobile app
- 5.5 hours Worked on Tensorflow Lite integration of mobile app- fixed bugs and refactored image pre-processing
- 4.5 hours Mobile app can now use test model to classify people
- 2.5 hours Some minor tweaking to app
- 4.0 hours Android app now supports affine transformation of input images
- 1.0 hour Test model now assumes previous affine transformation
- 4.5 hours Worked on YOLO TF model
- 4.0 hours Worked on YOLO TF model
- 5.0 hours Worked on YOLO TF model- model architecture is complete, many bugs remain
- 4.0 hours Worked on YOLO TF model (data loading)
- 5.0 hours Debugged YOLO TF model
- 3.0 hours Debugged YOLO TF model
- 4.5 hours Debugged YOLO TF model
- 1.5 hours Debugged YOLO TF model
- 2.5 hours Debugged YOLO TF model
- 3.5 hours Debugged YOLO TF model; model is working on MNIST dataset
- 2.0 hours Tried applying YOLO TF model to KAIST dataset (unsatisfactory results)
- 4.0 hours Tried applying YOLO TF model to KAIST dataset.Work will have to be discontinued due to extended problems (unsatisfactory results, difficulty of debugging, lack of resources to train the model for a sufficient number of epochs, difficulty of deploying the model to TF Lite)
- 2.0 hours Worked on implementing the NTG (normalised total gradient) metric to quantify image alignment/registration
- 3.0 hours Attempted other metrics for image alignment. Neither NTG nor any other metrics attempted yield good results.
- 1.0 hour Started writing dissertation chapter about image alignment
- 0.5 hours 9th supervisor meeting
- 1.0 hours Added evidence and conclusion to dissertation cahpter about image alignment
- 6.0 hours Created autoencoder to predict the FIR images of humans from the grayscale version of the RGB images. The model is working very well with a dataset consisting of 2 people.
- 4.0 hours Tried improving autoencoder by adding more convolutional layers. This was not successful.
- 1.5 hours Worked on visualization of autoencoder architecture
- 5.0 hours Implemented a texture-based image registration metric
- 3.0 hours Further investigation of texture-based image registration
- 1.5 hours Minor changes to autoencoder
- 3.0 hours Dissertation section on texture-based registration
- 2.0 hours Dissertation section on spectral imaging
- 5.0 hours Added more background sections to dissertation
- 3.0 hours Captured animal dataset with 12 classes and over 700 samples.
- 3.0 hours Pre-processed dataset (downsampling and label generation)
- 0.5 hours 10th supervisor meeting
- 2.0 hours Started implementing a classifier for dataset
- 1.0 hour Implemented residual block in Keras
- 1.0 hour Captured images of geese and cats to be added to dataset
- 1.5 hours Worked on dissertation chapter about data augmentation
- 4.0 hours Implemented affine data augmentation for dataset
- 2.0 hours Fixed issues with augmentation and regenerated dataset
- 4.0 hours Trained autoencoder on animals dataset
- 1.0 hour Created U-Net based on autoencoder and trained on subset of animals dataset
- 1.5 hours Evaluated autoencoder and U-Net on animal images from the internet
- 0.5 hours 11th supervisor meeting
- 2.0 hours Fixed class balancing in data augmentation pipeline
- 1.0 hour Re-generated augmented dataset
- 2.0 hours Successfully ran simple NN on augmented dataset. It appears as if visible light only performs better than VIS+LWIR. Further investigation is needed.
- 3.0 hour Started analysis section in dissertation
- 1.5 hours Dissertation: mobile application analysis
- 2.0 hours Implemented a variation of AlexNet
- 3.0 hours Familiarised myself with running jobs on the GPU cluster. Managed to run a simple classification job.
- 2.5 hours Set up custom Docker image for GPU cluster
- 3.0 hours Captured second animals dataset. This is to be able to perform a representative train/test split.
- 2.0 hours Added and refactored various pre-processing scripts from test notebooks. These will make it easier to run various tasks on the cluster.
- 1.0 hour Dissertation: dataset
- 2.0 hours Dissertation additions
- 2.0 hours Performed various tests on combined datasets. Models appear to struggle with chicken and alpacas.
- 1.0 hours Various improvements to pre-processing scripts
- 1.0 hours Experiments with subsets of dataset (e.g. chickena and alpacas only).
- 1.0 hours Evaluated different data augmentation techniques
- 2.0 hours Dissertation additions
- 0.5 hours 12th supervisor meeting
- 3.0 hours Captured third animals dataset, as classification accuracy was still too low with the existing data.
- 2.0 hours Created train/validation split from all 3 raw datasets. Started testing classification on Colab.
- 1.5 hours Refactored code for models.
- 1.5 hour Created a grid-search script for running 5 different configurations of models.
- 1.0 hours Added custom implementation of a residual network to models file.
- 3.0 hours Ran grid search for AlexNet and ResNet. ResNet seems to perform better overall. RGB-only is better than LWIR-only, but combined models are better than both.
- 0.5 hours 13th supervisor meeting
- 3.5 hours Dissertation additions
- 2.0 hours Added pre-made ResNet with pre-trained weights to grid search
- 0.5 hours Started grid-search on off-the-shelf Resnet
- 1.5 hours Miscellaneous additions to dissertation
- 2.0 hours Retrieved off-the-shelf dataset from FLIR One and wrote scripts to be able to load it.
- 3.0 hours Trained ResNet50 on FLIR One dataset. Prepared transfer-training the model on the custom dataset.
- 0.5 hours Added custom CNN to grid search
- 1.5 hours Added script for transfer learning from saved weights
- 2.0 hours Determined 95 reference points from the train dataset. Determined a new transformation matrix for registration based on these points.
- 1.5 hours Added registration results and evalution to dissertation
- 2.0 hours Prepared first incomplete draft of dissertation
- 0.5 hours Updated docker image to include easydict dependency
- 2.0 hours Added transfer learning to grid search
- 0.5 hours Minor restructuring of directory structure
- 0.5 hours 14th supervisor meeting
- 2.0 hours Added data augmentation to FLIR One dataset loader. Re-trained model on FLIR One dataset.
- 2.0 hours Various fixes
- 5.0 hours Dissertation additions (classifiers and autoencoder)
- 4.0 hours Prepared various visualisations for the dissertation
- 2.0 hours Successfully deployed RGB-only ResNet to mobile device. It seems to work (tested on pictures). Deployment of other modes should be trivial, although evaluation will be more difficult.
- 2.5 hours Various dissertation additions
- 1.5 hours Plotted loss and accuracy of grid search results. All metrics are very unstable and hardly interpretable.
- 3.5 hours Experimented with different batch sizes and learning rates to fix the problem mentioned above. Reducing learning rates and increasing batch sizes seems to help a lot.
- 2.0 hours Dissertation section on overfitting and regularisation
- 2.0 hours Batch size evaluation (including dissertation)
- 4.0 hours Dissertation: Mobile app eval and train-test-split eval
- 0.5 hours 15th supervisor meeting
- 6.0 hours Dissertation: Literature review
- 3.0 hours Ran final grid search on IDA cluster
- 5.0 hours Dissertation: Modality analysis
- 4.5 hours Dissertation: Started evaluation section
- 4.0 hours t-SNE and dissertation section
- 0.5 hours 16th supervisor meeting
- 3.0 hours Grad-CAM and dissertation section
- 2.5 hours K-fold cross-validation
- 6.0 hours Dissertation: Various additions
- 1.0 hours Fixed affine transformations (higher res to minimise interpolation errors)
- 3.0 hours Dissertation: Autoencoder evaluation
- 5.5 hours Dissertation: Abstract, introduction, various additions
- 4.5 hours Dissertation: Various additions
- 6.0 hours Dissertation: Various additions
- 0.5 hours 17th supervisor meeting
- 6.0 hours Dissertation review and quality improvements
- 5.5 hours Dissertation review and quality improvements
- 1.5 hours Started presentation
- 4.0 hours Dissertation review and quality improvements
- 3.0 hours Additions to presentation
- 1.0 hour Tidied up repo
- 3.0 hours Recorded presentation