Skip to content
This repository has been archived by the owner on Jun 21, 2024. It is now read-only.

Evaluate performance on few shot for an unseen tool after training toolformer #14

Open
dmahan93 opened this issue Feb 26, 2023 · 0 comments

Comments

@dmahan93
Copy link
Collaborator

Should compare with a few items:

Generation with few-shot on base model, same as data generation
Generating with few-shot on base model, with similar setup to inference
Generation with few-shot on trained model

Not sure if there should be an additional comparison with data generation hyperparams (pull out API calls > 10%, evaluate k calls, M shot) for the trained toolformer, as well.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant