New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3p][python]add metering and error details to 3p outputs #2143

Merged

lanking520 merged 1 commit into deepjavalibrary:master from siddvenk:3p-handler

Jul 8, 2024

Contributor

siddvenk commented Jul 3, 2024

Description

3p Changes:

Fix the output formatters to respond with the correct expected payload schema
Add metering and error details to the output schema for 3p use-case
force do_sample to be true when temperature > 0

General Change:

Add exception details to the Token that is set when rolling batch inference occurs. This makes it possible for output formatters to further provide specific error details in the response rather than just saying "error". This is currently only used by the 3p output formatters.


[3p][python]add metering and error details to 3p outputs

6fc25c6

siddvenk requested review from zachgk, frankfliu and a team as code owners

July 3, 2024 22:20

sindhuvahinis approved these changes

View reviewed changes

lanking520 merged commit 04445f8 into deepjavalibrary:master

9 checks passed

siddvenk deleted the 3p-handler branch

August 5, 2024 23:49

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment