InkSight: Offline-to-Online Handwriting Conversion by Learning to Read and Write

Blagoj Mitrevski† • Arina Rak† • Julian Schnitzler† • Chengkun Li† • Andrii Maksai‡ • Jesse Berent • Claudiu Musat

^† Joint first authors   |   ^‡ Corresponding author

---

News

• October 2024: We release Small-p on Hugging Face!

• October 2024: Our work is now featured on the Google Research Blog!

• February 2024: The InkSight Demo on Hugging Face is live!

---

Run Gradio 🤗 Playground Locally

To set up and run the Gradio Playground locally, you can use the following steps:

# Clone the huggingface space
git clone https://huggingface.co/spaces/Derendering/Model-Output-Playground

# Install the dependencies (skip if you have them already)
pip install gradio gdown

Then inside the cloned directory, you can run the following command to interact with the playground:

# Run the Gradio Playground
python app.py

---

Releases

⚠️ Notice: Please use TensorFlow and tensorflow-text between version 2.15.0 and 2.17.0. Versions later than 2.17.0 may lead to unexpected behavior. We are currently investigating these issues.

We provide open resources for InkSight public version model. Choose the options that best fit your needs:

• Public version model for CPU/GPU inference (494 MB)
• Hugging Face model for CPU/GPU inference: InkSight Small-p.
• Public version model for TPU inference (494 MB)
• Supplementary material for the paper. This is used in the example colab linked below, which automatically downloads this content.
• Example code in the form of a Colab notebook that showcases model inference results on several samples and example code to run the inference.
• Samples of model outputs of huggingface demo.

---

Licenses

The code in this repository is released under the Apache 2 license.

---

Disclaimer

Please note: This is not an officially supported Google product.

---

Citation

If you find our work useful for your research and applications, please cite using this BibTeX:

@article{mitrevski2024inksight,
  title={InkSight: Offline-to-Online Handwriting Conversion by Learning to Read and Write},
  author={Mitrevski, Blagoj and Rak, Arina and Schnitzler, Julian and Li, Chengkun and Maksai, Andrii and Berent, Jesse and Musat, Claudiu},
  journal={arXiv preprint arXiv:2402.05804},
  year={2024}
}