Skip to content

Commit

Permalink
Update README with HuggingFace content
Browse files Browse the repository at this point in the history
  • Loading branch information
icppWorld committed Apr 30, 2024
1 parent 4febd2e commit 5abf1de
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 17 deletions.
21 changes: 6 additions & 15 deletions llama2_c/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ For quick tests, we have included a really small model, with only 260K parameter
The CI/CD using a GitHub actions workflow, and the demo_pytest.sh script are based on this model.
# demo_pytest.sh
- The demo_pytest.sh script starts the local network, deploys llama2_260K, uploads the model & tokenizer, and runs the QA with pytest:
Expand All @@ -161,25 +162,15 @@ The CI/CD using a GitHub actions workflow, and the demo_pytest.sh script are bas
- `./demo.sh` , on Linux / Mac
- `.\demo.ps1` , in Windows PowerShell (Miniconda recommended)
# More models
# Models
- You can get other model checkpoints, as explained in [karpathy/llama2.c](https://github.com/karpathy/llama2.c):
## HuggingFace
This command downloads the 15M parameter model that was trained on the TinyStories dataset (~60MB download) and stores it in a `models` folder:
You can find many models in the llama2_c *.bin format on HuggingFace, for example:
```bash
# on Linux/Mac
mkdir -p models
wget -P models https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin
```
- [onicai/llama2_c_canister_models](https://huggingface.co/onicai/llama2_c_canister_models)
- [karpathy/tinyllamas](https://huggingface.co/karpathy/tinyllamas)
```powershell
# in Windows PowerShell (Miniconda recommended)
if (-not (Test-Path -Path .\models)) {
New-Item -Path .\models -ItemType Directory
}
Invoke-WebRequest -Uri https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin -OutFile .\models\stories15M.bin
```
# Deploying to the IC main net
Expand Down
6 changes: 4 additions & 2 deletions llama2_c/scripts/nft_update_story.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,12 +156,14 @@ def main() -> int:
print(response)
if "Ok" in response[0].keys():
# Check if the number of generated tokens is less than the requested tokens
if response[0]["Ok"]["num_tokens"] < prompt['steps']:
if response[0]["Ok"]["num_tokens"] < prompt["steps"]:
print(f'The end! - num_tokens = {response[0]["Ok"]["num_tokens"]}')
break
# Check if the response is an empty string. If it is, break out of the loop.
if response[0]["Ok"]["inference"] == "":
print("The end! - we got an empty string. THIS IS AN ERROR ACTUALLY. WE SHOULD NOT GET HERE..")
print(
"The end! - we got an empty string. (ERROR: WE SHOULD NOT GET HERE)"
)
print("Something went wrong:")
sys.exit(1)
else:
Expand Down

0 comments on commit 5abf1de

Please sign in to comment.