Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: #3698 - o1 preview models do not work with max_tokens #3728

Merged
merged 1 commit into from
Sep 24, 2024

Conversation

louis-jan
Copy link
Contributor

@louis-jan louis-jan commented Sep 24, 2024

Describe Your Changes

This PR addresses the issue where OpenAI preview models don't work with the max_tokens parameter. We'll likely deprecate model.json for remote models at least, so we're not introducing new model parameter support. Instead, we're using the standard payload transform that's been applied to other providers.

The changes are mostly about adding tests.

When making a request using o1-preview or o1-mini, getting:

Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.
Jan’s in beta. Access troubleshooting assistance now.

Cannot change these model parameters settings.

o1-preview
64707
gpt-3.5-turbo
45537

Fixes Issues

Additions and Changes:

  1. Addition of Jest Configuration:

    • A new Jest configuration file jest.config.js has been added under extensions/inference-openai-extension. This setting includes configurations such as preset, testEnvironment, and custom transformations for testing @janhq/core modules.
  2. Addition of Jest Test File:

    • A new Jest test file OpenAIExtension.test.ts has been added under src. This test suite mocks the @janhq/core module and includes tests for the JanInferenceOpenAIExtension class, including initialization, payload transformation, and other functionalities specific to different model types.
  3. Modifications to index.ts:

    • The index.ts file has been modified to incorporate new exports and enums, and introduce a transformPayload method within the JanInferenceOpenAIExtension class. The payload transformation accounts for differences between preview models (o1-mini, o1-preview) and non-preview models.
    • The payload transformation specifically handles fields like max_completion_tokens for preview models, while other models retain their original payload structure.
  4. Changes to tsconfig.json:

    • The TypeScript configuration file in tsconfig.json adds an "exclude" field to exclude test files (those ending in .test.ts) from compilation. This likely ensures that unit test files are not included in the final production build.

@github-actions github-actions bot added the type: bug Something isn't working label Sep 24, 2024
Copy link
Contributor

Barecheck - Code coverage report

Total: 62.56%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@louis-jan louis-jan merged commit acd3be3 into dev Sep 24, 2024
21 checks passed
@louis-jan louis-jan deleted the fix/3698-o1-preview-doesnot-work-with-max_tokens branch September 24, 2024 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
2 participants