#

chinese-tokenizer

Here are 8 public repositories matching this topic...

lionsoul2014 / friso

High performance Chinese tokenizer with both GBK and UTF-8 charset support based on MMSEG algorithm developed by ANSI C. Completely based on modular implementation and can be easily embedded in other programs, like: MySQL, PostgreSQL, PHP, etc.

c tokenizer full-text-search chinese-word-segmentation chinese-tokenizer php-tokenizer korean-tokenizer japanese-tokenizer cjk-tokenizer

Updated Oct 29, 2023
C

supercoderhawk / DeepLearning_NLP

基于深度学习的自然语言处理库

natural-language-processing deep-learning tensorflow named-entity-recognition relation-extraction chinese-word-segmentation chinese-tokenizer

Updated Nov 3, 2018
Python

howl-anderson / MicroTokenizer

一个轻量且功能全面的中文分词器，帮助学生了解分词器的工作原理。MicroTokenizer: A lightweight Chinese tokenizer designed for educational and research purposes. Provides a practical, hands-on approach to understanding NLP concepts, featuring multiple tokenization algorithms and customizable models. Ideal for students, researchers, and NLP enthusiasts..

tokenizer chinese-nlp nlp-machine-learning chinese-word-segmentation chinese-tokenizer educational-project dag-network

Updated Oct 18, 2024
Python

howl-anderson / Chinese_tokenizer_benchmark

中文分词软件基准测试 | Chinese tokenizer benchmark

tokenizer chinese-tokenizer benchmark-tests python-tokenizer

Updated Sep 5, 2018
Python

samzshi0529 / HanziNLP

A NLP package for Chinese text：Preprocessing, Tokenization, Chinese Fonts, Word Embeddings, Text Similarity and Sentiment Analysis 轻量级中文自然语言处理软件包

nlp machine-learning sentiment-analysis chinese hanzi nlp-machine-learning tokenization hanzi-pinyin chinese-tokenizer

Updated Nov 3, 2024
Python

howl-anderson / PaddleTokenizer

使用 PaddlePaddle 实现基于深度神经网络的中文分词引擎 | A DNN Chinese Tokenizer by Using PaddlePaddle

nlp machine-learning neural-network tokenizer chinese paddle chinese-tokenizer paddlepaddle tokenizer-framework nature-language-process

Updated Jul 27, 2020
JavaScript

volgachen / Chinese-Tokenization

UCAS Homework for NLP

nlp tokenization chinese-tokenizer

Updated Jul 11, 2020
Python

HuangStomach / the-imp

Chinese tokenizer base on nodejieba and pullword

tokenizer chinese-word-segmentation chinese-tokenizer nodejieba pullword

Updated Jan 18, 2024
JavaScript

Improve this page

Add a description, image, and links to the chinese-tokenizer topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the chinese-tokenizer topic, visit your repo's landing page and select "manage topics."