Skip to content

Commit

Permalink
Llama3 and phi3 validation result update. (opea-project#79)
Browse files Browse the repository at this point in the history
* Llama3 and Phi3 validation results update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
yao531441 and pre-commit-ci[bot] committed May 30, 2024
1 parent d9c4441 commit 3986c4f
Showing 1 changed file with 11 additions and 0 deletions.
11 changes: 11 additions & 0 deletions comps/llms/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,14 @@ curl http://${your_ip}:9000/v1/chat/completions \
-d '{"query":"What is Deep Learning?","max_new_tokens":17,"top_k":10,"top_p":0.95,"typical_p":0.95,"temperature":0.01,"repetition_penalty":1.03,"streaming":true}' \
-H 'Content-Type: application/json'
```

## Validated Model

| Model | TGI-Gaudi | vLLM-CPU | Ray |
| ------------------------- | --------- | -------- | --- |
| Intel/neural-chat-7b-v3-3 ||||
| Llama-2-7b-chat-hf ||||
| Llama-2-70b-chat-hf || - | x |
| Meta-Llama-3-8B-Instruct ||||
| Meta-Llama-3-70B-Instruct || - | x |
| Phi-3 | x | Limit 4K ||

0 comments on commit 3986c4f

Please sign in to comment.