Skip to content

irisliucy/Short-text-Classification

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Short Text Classification with Deep Neural Networks: An Exerimental Analysis

About

An experimental analysis with the aim to examine different neural networks and machine learning approaches. We also include a structural model that comprises of a temporal LSTM and a machine learning classifier. We extract the intermediate outputs from the hidden layers of a LSTM for feature engineering in a machine learning classifer.

We run our models on 3 different datasets, 20 Newsgroups dataset, Ohsumed dataset and Yahoo! Answer dataset. In order to solve a multi-class classification problem, we created benchmark datasets based on those source datasets.

Final Year Project work is available at: http://dspace.cityu.edu.hk/handle/2031/93

Requirements

Code is written in Python (2.7) and requires Keras (2.0.2) with theano backend.

Using the pre-trained Glove word embedding vectors (pre-trained on Twitter / Wikipedia corpus) will also require downloading the binary file from https://nlp.stanford.edu/projects/glove/

Using the GPUs

GPU is highly recommended to run the Keras models. We use two machines equipped with GeForce GTX 950 and took ~5-8 hours for a 20-class dataset. I believe you can speed things up with better GPUs.

To run on GPUs, you will need a theano.rc file in your home directory. Inside the file, you have to set device = gpu to enable the GPU.

[global]
device = gpu
floatX = float32
optimizer_including = cudnn
allow_gc = True

[nvcc]
flags=-D_FORCE_INLINES

[cuda]
root = /usr/local/cuda-8.0

[lib]
cnmem = 0.7

To choose Theano as Keras backend, you will need a Keras folder with a keras.json file. Insider the json file, you need to specify

{
    "image_dim_ordering": "th", 
    "epsilon": 1e-07, 
    "floatx": "float32", 
    "backend": "theano"
}

Models & Results

An overall result (in accuracy) of all models used in our experiments. The result is taken from 20-class, 23-class and 60-class of 20 News, Ohsumed and Yahoo datasets respectively.

Model 20 News Ohsumed Yahoo
CNN- 4 layer / 0.294767 0.622582
CNN- 7 layer 0.211398 / /
LSTM-rand 0.216380 / /
LSTM 0.719906 0.437263 0.689994
LSTM+SVM 0.772263 0.412098 0.759831
LSTM+KNN-Euclidean 0.748359 0.354187 0.746555
LSTM+KNN-DTW 0.90857 0.418788 0.712172
SVM 0.257188 0.17977 0.336113
KNN 0.264689 0.099642 0.336112
SVM-TFIDF 0.26634 0.18514 0.340016
KNN-TFIDF 0.271291 0.101608 0.335994

20News-All ohsumed-All Yahoo-All

About

Short Text Classification with Deep Neural Networks: An Experimental Analysis

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published