Skip to content

Commit

Permalink
feat(mamba): Add bagel-dpo-2.8b (#1671)
Browse files Browse the repository at this point in the history
Adds the Mamba-slimpj model fine-tuned with bagel.
https://huggingface.co/jondurbin/bagel-dpo-2.8b-v0.2

Signed-off-by: Richard Palethorpe <io@richiejp.com>
  • Loading branch information
richiejp authored Feb 2, 2024
1 parent 1c57f8d commit e46db63
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions embedded/models/mamba-bagel.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: bagel
backend: mamba
parameters:
model: "jondurbin/bagel-dpo-2.8b-v0.2"
systemPrompt: "You are a helpful, unbiased, uncensored assistant."
template:
chat_message: |
{{if eq .RoleName "assistant"}}{{.Content}}{{else}}
[INST]
{{if .SystemPrompt}}{{.SystemPrompt}}{{else if eq .RoleName "system"}}<<SYS>>{{.Content}}<</SYS>>
{{else if .Content}}{{.Content}}{{end}}
[/INST]
{{end}}
completion: |
{{.Input}}
usage: |
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "bagel",
"messages": [{"role": "user", "content": "how are you doing"}],
}'

0 comments on commit e46db63

Please sign in to comment.