This repository is for a german finetuned version of the LLaMA model. This model is designed for research use only.
** Currently training with llama 2 (31.08.2023) **
The Space is offline currently.
If I have seen further it is by standing on the back of a llama. -- Isaac Newton
Thanks to the portuguese finetuned version of LLaMA, Cabrita, for the inspiration of this project. Also thanks to Meta AI for developing and trainig the base LLaMA model and sparking the era of finetuning, apply for the LLaMA model weights here.
A german-translated version of the cleaned Stanford Alpaca dataset was used. The original Stanford Alpaca dataset is available here. The translation was done via the facebook/wmt19-en-de model from HuggingFace 🤗 Hub.
More information about how the dataset is configured can be found here: Stanford Alpaca.
The baseline LLaMA-7b model was finetuned using the LoRA method in 8bit quantization, using parts of the code from Alpaca Lora fork, which makes use of PEFT from HuggingFace 🤗. The finetuning script is located here. The finetuning was done using a single A100 40GB GPU for 1000 steps.
The finetuned model checkpoints can be downloaded on HuggingFace 🤗:
Stanford Alpaca:
In welchen Ländern gibt es Lamas?
Llamas can be found in Bolivia, Peru, Chile, Ecuador, and Argentina.
Zicklein:
In welchen Ländern gibt es Lamas?
Lamas gibt es in den folgenden Ländern: Argentinien, Bolivien, Brasilien, Chile, Ecuador, Kolumbien,
Peru, Uruguay und Venezuela.
Stanford Alpaca:
Was ist die Rolle des Präsidenten?
The role of the President is to serve as the head of state and the commander-in-chief of the armed
forces. The President is responsible for executing the laws of the land, appointing judges and other
government officials, and representing the United States in international relations.
Zicklein:
Was ist die Rolle des Präsidenten?
Die Rolle des Präsidenten besteht darin, die Verfassung der Vereinigten Staaten zu verwalten und die
Regierung zu leiten.
- Code: Apache License 2.0
- Dataset: CC BY-NC 4.0
- Model: Apache License 2.0