support llama2 13b train and inference pipeline in fastchat #64
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ruiyiw
force-pushed
the
feature/llama2-13b-train
branch
from
52153ed to
daa6c59
Compare
lwaekfjlk
changed the title
lwaekfjlk
reviewed
Oct 24, 2023
llm_ft/fastchat/train/train.py
Outdated
| @@ -109,6 +122,15 @@ def preprocess( | |||
| max_length=tokenizer.model_max_length, | |||
| truncation=True, | |||
| ).input_ids | |||
| # print(input_ids.size()) | |||
There was a problem hiding this comment.
delete those comments to make it clean
| @@ -54,16 +54,19 @@ def match(self, model_path: str): | |||
|
|
|||
| def load_model(self, model_path: str, from_pretrained_kwargs: dict): | |||
| revision = from_pretrained_kwargs.get("revision", "main") | |||
| print(from_pretrained_kwargs) | |||
There was a problem hiding this comment.
delete this print line of code
lwaekfjlk
reviewed
Oct 24, 2023
llm_ft/data/data_filter_out_long.py
Outdated
| data = json.load(f) | ||
|
|
||
| tokenizer = transformers.AutoTokenizer.from_pretrained( | ||
| 'meta-llama/Llama-2-13b-chat-hf', |
There was a problem hiding this comment.
make it a model name instead of llama-specific
llm_ft/inference.sh
Outdated
| python3 -m fastchat.serve.cli --model-path ./checkpoint-shuffle/checkpoint-161 --hf-access-token "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" --conv-template "vicuna_v1.1" |
There was a problem hiding this comment.
change this to llama-inference.sh
lwaekfjlk
pushed a commit
that referenced
this pull request
Nov 17, 2023
* add 2qa * save * change prompt * eval v2 * add tables * add reviewer, prompt * add reviews * rename * tables * new line * update * update * rename
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #50
📑 Description
Added option of Hugging Face access token to use llama2 directory from hungging face.
Added option of shuffling the dataset before training.
Added script for generating dummy data to test fine-tune validity.
Modified train shell script for llama-2 experiments.
✅ Checks
type/descript(e.g.feature/add-llm-agents)ℹ Additional Information