Skip to content

Latest commit

 

History

History
14 lines (8 loc) · 879 Bytes

File metadata and controls

14 lines (8 loc) · 879 Bytes

LSTM-Language-Model-with-Noise-Contrastive-Estimation

A Language Model built using Long-Short Term Memory (LSTM) Recurrent Neural Network (RNN) for solving the problem of vanishing gradients. Noise Contrastive Estimation (NCE) is used to avoid the softmax normalization at the output layer, which reduces the computation time. Theano and Lasagne are used for building the language model.

The training model will be saved in the 'models' folder after the python script is run.

I personally use Anaconda, which installs most of the packages that one may require for coding in Python. Still, if you want to install Theano and Lasagne manually, the links for step-by-step installation are provided below.

Theano installation: -

http://deeplearning.net/software/theano/install_ubuntu.html

Lasagne installation: -

http://lasagne.readthedocs.io/en/latest/user/installation.html