Skip to content

Commit

Permalink
docs: update function docs
Browse files Browse the repository at this point in the history
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
  • Loading branch information
mudler authored May 10, 2024
1 parent 9b09eb0 commit 88d0aa1
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions docs/content/docs/features/openai-functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,28 @@ When running the python script, be sure to:

## Advanced

### Use functions without grammars

The functions calls maps automatically to grammars which are currently supported only by llama.cpp, however, it is possible to turn off the use of grammars, and extract tool arguments from the LLM responses, by specifying in the YAML file `no_grammar` and a regex to map the response from the LLM:

```yaml
function:
no_grammar: true
response_regex: "..."
```
The response regex have to be a regex with named parameters to allow to scan the function name and the arguments. For instance, consider:
```
(?P<function>\w+)\s*\((?P<arguments>.*)\)
```

will catch

```
function_name({ "foo": "bar"})
```

### Parallel tools calls

This feature is experimental and has to be configured in the YAML of the model by enabling `function.parallel_calls`:
Expand Down

0 comments on commit 88d0aa1

Please sign in to comment.