NLP-Papers

• Distributed Word Representations
• Distributed Sentence Representations
• Entity Recognition (Sequence Tagging)
• Language Model (LM for pre-training)
• Machine Translation
• Question Answering (Machine Reading Comprehension)
• Recommendation Systems
• Relation Extraction
• Sentences Matching (Natural Language Inference/Textual Entailment)
• Text Classification (Sentiment Classification)
• Materials/Toolkits/Corpus

Papers and Notes

Distributed Word Representations

• 2017-11
  • Faruqui and Dyer - 2014 - Improving vector space word representations using multilingual correlation [pdf] [note]
  • Maaten and Hinton - 2008 - Visualizing data using t-SNE [pdf] [pdf (annotated)] [note]
  • Ling et al. - 2015 - Finding function in form: Compositional character models for open vocabulary word representation [pdf] [pdf (annotated)] [note]
  • Bojanowski et al. - 2016 - Enriching word vectors with subword information [pdf] [pdf (annotated)] [note]
• 2017-12
  • Bengio and Senécal - 2003 - Quick Training of Probabilistic Neural Nets by Importance Sampling [pdf] [pdf(annotated)] [note]
• references
  • word2vec(tensorflow)
  • subword-based word vector
  • Chinese Word Vectors 中文词向量
  • Tencent AI Lab Embedding Corpus for Over 8 Million Chinese Words and Phrases

Distributed Sentence Representations

• 2017-11
  • Le and Mikolov - 2014 - Distributed representations of sentences and documents [pdf] [pdf (annotated)] [note]
• 2018-12
  • Li and Hovy - 2014 - A Model of Coherence Based on Distributed Sentence Representation [pdf] [pdf (annotated)] [note]
  • Kiros et al. - 2015 - Skip-Thought Vectors [pdf] [pdf (annotated)] [note]
  • Hill et al. - 2016 - Learning Distributed Representations of Sentences from Unlabelled Data [pdf] [pdf (annotated)] [note]
  • Arora et al. - 2016 - A simple but tough-to-beat baseline for sentence embeddings [pdf] [pdf (annotated)] [note]
  • Pagliardini et al. - 2017 - Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features (sent2vec) [pdf] [pdf (annotated)] [note]
  • Logeswaran et al. - 2018 - An efficient framework for learning sentence representations (Quick-Thought Vectors) [pdf] [pdf (annotated)] [note]
• 2019-01
  • Wieting et al. - 2015 - Towards universal paraphrastic sentence embeddings [pdf] [pdf (annotated)] [note]
  • Adi et al. - 2016 - Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks [pdf] [pdf (annotated)] [note]
  • Conneau et al. - 2017 - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data (InferSent) [pdf] [pdf (annotated)] [note]
  • Cer et al. - 2018 - Universal Sentence Encoder [pdf] [pdf (annotated)] [note]
• references
  • awesome-sentence-embedding: A curated list of pretrained sentence(and word) embedding models
  • SentEval: evaluation toolkit for sentence embeddings
  • doc2vec(gensim)
  • Skip-Thought Vectors
  • SIF(sentence embedding by Smooth Inverse Frequency weighting scheme)
  • Quick-Thought Vectors
  • sent2vec
  • InferSent

Entity Recognition

• 2018-10
  • Lample et al. - 2016 - Neural Architectures for Named Entity Recognition [pdf]
  • Ma and Hovy - 2016 - End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF [pdf]
  • Yang et al. - 2017 - Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks [pdf]
  • Peters et al. - 2017 - Semi-supervised sequence tagging with bidirectional language models [pdf]
  • Shang et al. - 2018 - Learning Named Entity Tagger using Domain-Specific Dictionary [pdf]
• references
  • ChineseNER (TensorFlow)
  • flair (PyTorch)

Language Model

• 2017-11
  • Bengio et al. - 2003 - A neural probabilistic language model [pdf]
  • Press and Wolf - 2016 - Using the output embedding to improve language model [pdf]
• 2019-02
  • Peters et al. - 2018- Deep contextualized word representations(ELMo) [pdf] [note]
  • Howard and Ruder - 2018 - Universal language model fine-tuning for text classification(ULMFit) [pdf]
  • Radford et al. - 2018 - Improving language understanding by generative pre-training [pdf]
  • Devlin et al. - 2018 - Bert: Pre-training of deep bidirectional transformers for language understanding [pdf]
• references
  • Blog:The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
  • ELMo
    • ELMo(AllenNLP)
    • Pre-trained ELMo Representations for Many Languages
  • Quick Start: Training an IMDb sentiment model with ULMFiT
  • finetune-transformer-lm: Code and model for the paper "Improving Language Understanding by Generative Pre-Training"
  • BERT
    • google-research/bert: officical TensorFlow code and pre-trained models for BERT
    • huggingface/transformers provides state-of-the-art general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet, CTRL...) for NLU and NLG using TensorFlow 2.0 and PyTorch.
    • awesome-bert: bert nlp papers, applications and github resources ， BERT 相关论文和 github 项目

Machine Translation

• 2017-12
  • Oda et al. - 2017 - Neural Machine Translation via Binary Code Predict [pdf] [note]
  • Kalchbrenner et al. - 2016 - Neural machine translation in linear time [pdf] [pdf (annotated)] [note]
• 2018-05
  • Sutskever et al. - 2014 - Sequence to Sequence Learning with Neural Networks [pdf]
  • Cho et al. - 2014 - Learning Phrase Representations using RNN Encoder-Decoder for NMT [pdf]
  • Bahdanau et al. - 2014 - NMT by Jointly Learning to Align and Translate [pdf]
  • Luong et al. - 2015 - Effective Approaches to Attention-based NMT [pdf]
• 2018-06
  • Gehring et al. - 2017 - Convolutional sequence to sequence learning [pdf]
  • Vaswani et al. - 2017 - Attention is all you need [pdf] [note1:The Illustrated Transformer] [note2:The Annotated Transformer]
• references
  • OpenNMT-py (in PyTorch)
  • nmt (in TensorFlow)
  • MT-Reading-List

Question Answering

• 2018-03
  • Wang and Jiang. - 2016 - Machine Comprehension Using Match-LSTM and Answer Pointer [pdf]
  • Seo et al. - 2016 - Bidirectional Attention Flow for Machine Comprehension [pdf]
  • Cui et al. - 2016 - Attention-over-Attention Neural Networks for Reading Comprehension [pdf]
• 2018-04
  • Clark and Gardner. - 2017 - Simple and Effective Multi-Paragraph Reading Comprehension [pdf]
  • Wang et al. - 2017 - Gated Self-Matching Networks for Reading Comprehension and Question Answering [pdf]
  • Yu et al. - 2018 - QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension [pdf]
• references
  • DuReader
  • SQuAD
  • MS MARCO
  • 深度学习解决机器阅读理解任务的研究进展
  • RCPapers: Must-read papers on Machine Reading Comprehension

Recommendation Systems

• 2019-05
  • Rendle S. - 2010 - Factorization machines [pdf] [note]
  • Cheng et al. - 2016 - Wide & Deep Learning for Recommender Systems [pdf]
  • Guo et al. - 2017 - DeepFM: A Factorization-Machine based Neural Network for CTR Prediction [pdf]
  • He and Chua. - 2017 - Neural Factorization Machines for Sparse Predictive Analytics [pdf]
• references
  • 谷歌、阿里、微软等10大深度学习CTR模型最全演化图谱【推荐、广告、搜索领域】

Relation Extraction

• 2018-08
  • Mintz et al. - 2009 - Distant supervision for relation extraction without labeled data [pdf]
  • Zeng et al. - 2015 - Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks [pdf]
  • Zhou et al. - 2016 - Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification [pdf]
  • Lin et al. - 2016 - Neural Relation Extraction with Selective Attention over Instances [pdf]
• 2018-09
  • Ji et al. - 2017 - Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions [pdf]
  • Levy et al. - 2017 - Zero-Shot Relation Extraction via Reading Comprehension [pdf]
• references
  • OpenNRE
  • NREPapers: Must-read papers on neural relation extraction (NRE)
  • awesome-relation-extraction
  • Entity-Relation-Extraction(管道式实体及关系抽取)

Sentences Matching

• 2017-12
  • Hu et al. - 2014 - Convolutional neural network architectures for Matching Natural Language Sentences [pdf] [pdf (annotated)] [note]
• 2018-07
  • Nie and Bansal - 2017 - Shortcut-Stacked Sentence Encoders for Multi-Domain Inference [pdf] [note]
  • Wang et al. - 2017 - Bilateral Multi-Perspective Matching for Natural Language Sentences [pdf] [note]
  • Tay et al. - 2017 - A Compare-Propagate Architecture with Alignment Factorization for Natural Language Inference [pdf]
  • Chen et al. - 2017 - Enhanced LSTM for Natural Language Inference [pdf] [note]
  • Ghaeini et al. - 2018 - DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference [pdf]
• references
  • The Stanford Natural Language Inference (SNLI) Corpus
  • A curated list of papers dedicated to neural text (semantic) matching.
  • MatchZoo:一个通用的文本匹配工具包 [tensorflow] [pytorch]
  • AnyQ:面向FAQ集合的问答系统框架、文本语义匹配工具SimNet
  • Kaggle: Quora Question Pairs

Text Classification

• 2017-09
  • Joulin et al. - 2016 - Bag of tricks for efficient text classification [pdf] [pdf (annotated)] [note]
• 2017-10
  • Kim - 2014 - Convolutional neural networks for sentence classification [pdf] [pdf (annotated)] [note]
  • Zhang and Wallace - 2015 - A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification [pdf] [pdf (annotated)] [note]
  • Zhang et al. - 2015 - Character-level convolutional networks for text classification [pdf] [pdf (annotated)] [note]
  • Lai et al. - 2015 - Recurrent Convolutional Neural Networks for Text Classification [pdf] [pdf (annotated)] [note]
  • Yang et al. - 2016 - Hierarchical attention networks for document classification [pdf]
• 2017-11
  • Iyyer et al. - 2015 - Deep unordered composition rivals syntactic methods for Text Classification [pdf] [pdf (annotated)] [note]
• 2019-04 (Aspect level sentiment classification)
  • Wang et al. - 2016 - Attention-based LSTM for aspect-level sentiment classification [pdf]
  • Tang et al. - 2016 - Aspect level sentiment classification with deep memory network [pdf]
  • Chen et al. - 2017 - Recurrent Attention Network on Memory for Aspect Sentiment Analysis [pdf]
  • Xue and Li - 2018 - Aspect Based Sentiment Analysis with Gated Convolutional Networks [pdf]
• references
  • fastText
  • text_classification
  • PyTorchText(知乎看山杯)

Materials

• Neural Networks for NLP (CS11-747 Spring 2019 @ CMU)

• deep learning

  • An overview of gradient descent optimization algorithms
  • Focal Loss（缓解多分类样本不均衡问题）

• NLP-progress

• Awesome-Chinese-NLP

• StateOfTheArt.ai

• funNLP(从文本中抽取结构化信息等资源汇总)

• StanfordNLP: Official Stanford NLP Python Library for Many Human Languages

• Browse state-of-the-art

• Corpus

  • ChineseNLPCorpus
  • nlp_chinese_corpus
  • CLUE(中文任务基准测评)