The model is defined in conv_1d_LSTM.py. This model features a recurrent neural network (RNN) that consists of long short-term memory (LSTM) units, with a single fully connected layer at the top and optional 1D convolution and pooling layers at the bottom. The convolution and pooling layers can be added through the _LAYERS argument using the format described in the comment.
The script convert_files.py converts raw binary data files into a format that can be readily used for training and evaluation purposes. The program has the following command line arguments:
sensor_type: sensor type; must be between 1 and NUM_SENSOR_TYPES defined in run.py
--mode: must be one of "train", "validate", "predict"
--data_dir: directory in which the original data files are located; defaults to the same dir as the script
--num_sequences_per_file: the number of sequences to extract from each file; defaults to 5000
Once the arguments are specified, the program extracts args.num_sequences_per_file sequences from each file at random offsets and writes them to an output file named "<mode>_r/<sensor_type/>". This file name will be recognized by run.py or serve.py. Each extracted sequence has a fixed length specified by a constant from run.py and represents a training instance. A label (i.e., transmitter type 0 or 1) is added at the beginning of each sequence written to facilitate training and evaluation.
The script run.py trains and evaluates the model defined in conv_1d_LSTM.py using the files generated by convert_files.py. The program uses the tf.data.Dataset for creating input pipelines and the tf.estimator.Estimator classes for training and evaluating. The program has the following command line arguments:
sensor_type: sensor type; must be between 1 and NUM_SENSOR_TYPES defined in run.py
--data_format: "channels_first" or "channels_last"; defaults to "channels_last"
--batch_size: minibatch size during training; defaults to 200
--train_epochs: number of training epochs; defaults to 100
--epochs_between_evals: number of epochs between evaluations; defaults to 10
--max_train_steps: maximum number of training steps; defaults to 20000
--loss_scale: scaling factor for loss; defaults to 1
--data_dir, directory in which the converted data files are stored; defaults to the same dir as the script
When the program is run, two directories, "./model_r<sensor_type>" and "./export_r<sensor_type>" are created. Data related to computation graphs, tf.summary and checkpoints are saved under "./model_r<sensor_type>". When training completes, the trained model is saved as a SavedModel under "./export_r<sensor_type>" for future serving. If either of the directories already exists, the program prints an error message and exits. So make sure to delete or rename any existng directories with the same name before running the script.
The script serve.py loads a saved model from disk which generates predictions for new data and prints them out as a list in the console. The program has the following command line arguments:
sensor_type: sensor type; must be between 1 and NUM_SENSOR_TYPES defined in run.py
--data_dir: directory in which the data files are located; defaults to the same dir as the script
--export_dir: directory in which the trained model is located; defaults to the same dir as the script