Skip to content

Commit

Permalink
add optional kwargs to readme
Browse files Browse the repository at this point in the history
  • Loading branch information
zfletch committed Apr 8, 2023
1 parent 2246714 commit b881ba1
Showing 1 changed file with 24 additions and 6 deletions.
30 changes: 24 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,17 +40,35 @@ m.predict('hello world')
#### Llama::Model.new

```ruby
def self.new(
model, # path to model file, e.g. "models/7B/ggml-model-q4_0.bin"
n_predict: 128 # number of tokens to predict
seed: Time.now.to_i, # RNG seed
)
require 'llama'

Llama::Model.new('models/7B/ggml-model-q4_0.bin')
```

Optional arguments:

```ruby
seed # RNG seed (default Time.now.to_i)
n_predict # number of tokens to predict (default: 128, -1 = infinity)
threads # number of threads to use during computation (default: 4)
top_k # top-k sampling (default: 40)
top_p # top-p sampling (default: 0.9)
repeat_last_n # last n tokens to consider for penalize (default: 64)
repeat_penalty # penalize repeat sequence of tokens (default: 1.1)
ctx_size # size of the prompt context (default: 512)
ignore_eos # ignore end of stream token and continue generating
memory_f32 # use f32 instead of f16 for memory key+value
temp # temperature (default: 0.8)
n_parts # number of model parts (default: -1 = determine from dimensions)
batch_size # batch size for prompt processing (default: 8)
keep # number of tokens to keep from the initial prompt (default: 0, -1 = all)
mlock # force system to keep model in RAM rather than swapping or compressing
```

#### Llama::Model#predict

```ruby
def predict(prompt)
model.predict('hello world')
```

## Development
Expand Down

0 comments on commit b881ba1

Please sign in to comment.