Release 0.0.2 -- inference server & other improvements!

[francoishernandez]

We just released 0.0.2, which has been quite an iteration since 0.0.1.
Some long awaited changes are finally making it to the main branch!

🌟 Key Features

• Llama3.1 Support: We've boosted support for Llama3.1, including RoPE scaling and the introduction of mapped_tokens to efficiently handle specific prompt tokens.
• Inference Server: The first version of our inference server is live! This sets the foundation for more features and facilitated iteration for upcoming use cases. Check out the example recipe!
• Improved Configuration Management: Tokenization and inference-related settings are now embedded within the model configuration itself, making predictions and serving more seamless.

⚙️ Notable Improvements

• bfloat16 Support: Unlock better performance on specialized hardware with our new bfloat16 support, balancing memory use and precision.
• WMT22 Recipes: New recipes featuring TowerInstruct and Llama3.1 LLMs to simplify experimentation and setup.

💬 Drop a comment or get in touch if you have any feedback or question!