NNTransitionSegmentor is a package for Word Segmentation using neural networks based on package LibN3L.
The current version is a re-implementation of segmentor in ZPar.
What is the transition-based framework with beam-search decoding? Please see our ACL2014 tutorial: Syntactic Processing Using Global Discriminative Learning and Beam-Search Decoding
Take averaged perceptron as an example (CTB6.0, please refer to LibN3L: A lightweight Package for Neural NLP for details):
Both ZPar and this package obtain performance about 95.08%;
The normal sparse model, with max-margin training, reaches a F-measure of 95.24%.
- Download LibN3L library and compile it.
- Open CMakeLists.txt and change "../LibN3L/" into the directory of your LibN3L package.
cmake .
make
one line one sentence, with words seperated by spaces
- one can remove the length and keyChar embeddings in my implementation to reproduce the results of my ACL paper, because the two kinds of embeddings induce little influences in the final performances, but later experiments I found that this can be more stable.
- I will make the code more readable in the future. However if someone is interested in this framework, please concat me without hesitatation.