V2.0 implementation design

[BrikerMan]

Hello.

I have started the migration to tf2 for Kashgari V2.0. Here are the vision and roadmap.

Same Principle

V2.0 will be based on the same vision for V1.0, which is to create a clean and simple NLP framework for Academic users, NLP beginners, and Senior NLP developers.

• Human-friendly. Kashgari's code is straightforward, well documented, and tested, which makes it very easy to understand and modify.
• Powerful and simple. Kashgari allows you to apply state-of-the-art natural language processing (NLP) models to your text, such as named entity recognition (NER), part-of-speech tagging (PoS) and classification.
• Built-in transfer learning. Kashgari built-in pre-trained BERT and Word2vec embedding models, which makes it very simple to transfer learning to train your model.
• Fully scalable. Kashgari provides a simple, fast, and scalable environment for fast experimentation, train your models and experiment with new approaches using different embeddings and model structure.
• Production Ready. Kashgari could export model with SavedModel format for tensorflow serving, you could directly deploy it on the cloud.

RoadMap

• Based on TensorFlow 2.1+ [@BrikerMan]
• Fully support generator based training ([Feature request] 能否加入keras train_on_batch的api，大的数据集内存无法完整加载 #336 ,How to train on limited RAM when I have >10million documents #273) [@BrikerMan]
• Clean code and full document
• Multi GPU/TPU Support [@BrikerMan]
• Embeddings
  • Bare Embedding [@BrikerMan]
  • Word Embedding (Load trained W2V) [@BrikerMan]
  • BERT Embedding (Based on bert4keras, support BERT, RoBERTa, ALBERT...) ([Feature request] albert 是否可以加入albert模型 #316) [@BrikerMan]
  • GPT-2 Embedding
  • FeaturesEmbedding (Support Numeric feature as input)
  • Stacked Embedding (Stack Text embedding and features Embedding)
• Classification Task
  • Traditional models (CCN/LSTM/GRU) [@BrikerMan]
  • Transformer models (Transformer/Transformer XL/Attention)
• Labeling Task
  • Traditional models (CCN/LSTM/GRU)
  • Transformer models (Transformer/Transformer XL/Attention)
• Seq2Seq Task
  • Traditional models (CCN/LSTM/GRU)
  • Transformer models (Transformer/Transformer XL/Attention)
• Built-in Callbacks [@fxsld]
  • Evaluate Callback
  • Save Best Callback (How to save and load best model? [Question]  #331)
• Support TensorFlow Hub (Optional)

Contribution

If you are interested in implementing any part of the RoadMap or have any new ideas about Kashgari V2.0, feel free to comment.

[adlinex]

I'm willing to take Built-in callback module

[lsgrep]

I can do the Seq2Seq .