Skip to content

Commit

Permalink
Updated README Getting Started instructions
Browse files Browse the repository at this point in the history
Provides guidance to avoid error when downloading pre-trained model
  • Loading branch information
massenz committed Aug 19, 2023
1 parent d72c7fe commit c4177d2
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 3 deletions.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -160,4 +160,5 @@ cython_debug/
#.idea/

data/
wandb/
wandb/
.idea/
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,18 @@ First, we have to install all the libraries listed in `requirements.txt`
```bash
pip install -r requirements.txt
```

If you see this error:

> OSError: bigcode/starcoder is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

it means that you need to authenticate to Hugging Face API to download the model: sign up for an account, and accept the [T&C to use BigCode](https://huggingface.co/bigcode/starcoder); then [obtain an API Token](https://huggingface.co/settings/tokens) from HF and use it to authenticate to the CLI:

```shell
huggingface-cli login
```

## Code generation
The code generation pipeline is as follows

Expand All @@ -46,7 +58,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "bigcode/starcoder"
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
tokenizer = AutoTokenizer.from_pretrained(checkpoint, use_auth_token=True)
# to save memory consider using fp16 or bf16 by specifying torch_dtype=torch.float16 for example
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

Expand All @@ -60,7 +72,7 @@ or
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
checkpoint = "bigcode/starcoder"

model = AutoModelForCausalLM.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint, use_auth_token=True)
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)
Expand Down

0 comments on commit c4177d2

Please sign in to comment.