Skip to content

pytorch ntm python neural-network neural-turing-machines

Notifications You must be signed in to change notification settings

bombs-kim/visual_ntm

Repository files navigation

PyTorch Neural Turing Machines with Pretty Visualizer

Neural Turing Machines, arxiv:1410.5401

This is a NTM implementation in pytorch with a memory and read/write head visualizer. This project is intended to be instructional. The code is succinct and easy to understand. The visualizer helps you to grasp how heads and memory act with respect to an input. Currently, the copy task has been implemted and the other tasks are planned to be added.

Requirements

  • python3
  • pytorch 1.0


How to train

train.py [-h] [--sequence_length SEQUENCE_LENGTH]
                [--sequence_width SEQUENCE_WIDTH]
                [--num_memory_locations NUM_MEMORY_LOCATIONS]
                [--memory_vector_size MEMORY_VECTOR_SIZE]
                [--batch_size BATCH_SIZE] [--training_size TRAINING_SIZE]
                [--controller_output_size CONTROLLER_OUTPUT_SIZE]
                [--learning_rate LEARNING_RATE] [--momentum MOMENTUM]
                [--alpha ALPHA] [--min_grad MIN_GRAD] [--max_grad MAX_GRAD]
                [--weight_decay WEIGHT_DECAY] [--load LOAD] [--save SAVE]
                [--save_best SAVE_BEST]

How to test

python test.py --load pretrained

How to run visualizer

cd path/to/this/project
python -m http.server 8000

and open another terminal

cd path/to/this/project
python test.py --load pretrained --monitor_state

It is a good idea to use a debugger with the visualizer like in the demo.gif.


settings
  • sequence_length: 1~10
  • sequence_width: 8
  • num_memory_locations: 128
  • memory_vector_size: 20
  • batch_size: 1
  • controller_output_size: 100
Weight saved as pretrained file
whose mean loss is 0.00170

About

pytorch ntm python neural-network neural-turing-machines

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published