bpe
Here are 77 public repositories matching this topic...
PyTorch original implementation of Cross-lingual Language Model Pretraining.
-
Updated
Jul 28, 2020 - Python
An extremily simple and restricted tool/lib converting binary data into text that can be processed with unsuperwised character-level natural language processing tools/libs
-
Updated
Oct 13, 2023 - Python
-
Updated
Mar 1, 2023 - Shell
Java library implementing Byte-Pair Encoding Tokenization
-
Updated
May 17, 2023 - Java
simple chatbot using NLP and BPE
-
Updated
Jul 28, 2023 - Jupyter Notebook
A modified, secure version of BPE algorithm
-
Updated
Mar 29, 2024 - Python
This repository provides a clear, educational implementation of Byte Pair Encoding (BPE) tokenization in plain Python. The focus is on algorithmic understanding, not raw performance.
-
Updated
Aug 28, 2024 - Python
Source crypt Gradle plugin
-
Updated
May 3, 2022 - Kotlin
-
Updated
Sep 4, 2022 - Jupyter Notebook
Byte-Pair Encoding (BPE) (subword-based tokenization) algorithm implementaions from scratch with python
-
Updated
Jan 30, 2023 - Python
BPE (Byte-Pair Encoding) Encoder Decoder for OpenAI's GPT-2 / GPT-3 Implemented In Pure PHP, Zero Dependency, Multi Byte Supported.
-
Updated
Feb 27, 2023 - PHP
Information Retreival Course Repository, 7th Semester, IITD, 2023-24
-
Updated
Dec 11, 2023 - C
An educational project dedicated to text-to-image generation with neural networks. VQVAE and BPE autoencoders are used to learn the embedding of text and image respectively. A transformer-based model then is trained to predict the next token in the concatenated sequence of image and text tokens and used for generation.
-
Updated
Jun 8, 2021 - Python
Strings Tokenization with Byte Pair Encoding.
-
Updated
May 29, 2024 - TypeScript
Improve this page
Add a description, image, and links to the bpe topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the bpe topic, visit your repo's landing page and select "manage topics."