Skip to content

A Language Model built using Long-Short Term Memory (LSTM) Recurrent Neural Network (RNN) for solving the problem of vanishing gradients. Noise Contrastive Estimation (NCE) is used to avoid the softmax normalization at the output layer, which reduces the computation time.

License

Notifications You must be signed in to change notification settings

arnavks97/LSTM-Language-Model-with-Noise-Contrastive-Estimation

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LSTM-Language-Model-with-Noise-Contrastive-Estimation

A Language Model built using Long-Short Term Memory (LSTM) Recurrent Neural Network (RNN) for solving the problem of vanishing gradients. Noise Contrastive Estimation (NCE) is used to avoid the softmax normalization at the output layer, which reduces the computation time. Theano and Lasagne are used for building the language model.

The training model will be saved in the 'models' folder after the python script is run.

I personally use Anaconda, which installs most of the packages that one may require for coding in Python. Still, if you want to install Theano and Lasagne manually, the links for step-by-step installation are provided below.

Theano installation: -

http://deeplearning.net/software/theano/install_ubuntu.html

Lasagne installation: -

http://lasagne.readthedocs.io/en/latest/user/installation.html

About

A Language Model built using Long-Short Term Memory (LSTM) Recurrent Neural Network (RNN) for solving the problem of vanishing gradients. Noise Contrastive Estimation (NCE) is used to avoid the softmax normalization at the output layer, which reduces the computation time.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages