Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix] Partners integration tests #644

Closed
wants to merge 27 commits into from
Closed

[Fix] Partners integration tests #644

wants to merge 27 commits into from

Conversation

the-praxs
Copy link
Member

@the-praxs the-praxs commented Jan 16, 2025 •
edited
Loading

📥 Partners integration tests

Motivation

  • Adds integration tests for the partners in agentops
  • Fixes tests for the llm providers if they were skipped or flaky

Challenges

  • TaskWeaver has dependency issues with litellm over the jsonschema library version. litellm uses 4.20.00 while TaskWeaver requires 4.22.0
  • TaskWeaver has both a partner and provider integration so we need to add tests for both

Caveats

  • crewAI has bugs that need to be resolved in a separate PR
  • Llama Stack was not part of the original test suite hence is not addressed here

@the-praxs the-praxs marked this pull request as draft January 16, 2025 18:10
@teocns
Copy link
Contributor

teocns commented Jan 16, 2025

this drags commits from another branch that is not yet merged into main?

@the-praxs
Copy link
Member Author

the-praxs commented Jan 16, 2025 •
edited
Loading

this drags commits from another branch that is not yet merged into main?

Except feat/v0.4 - nope

@the-praxs the-praxs force-pushed the fix/partners-integration-tests branch from 9f7e992 to 9f72715 Compare January 16, 2025 18:30
@codecov Codecov
Copy link

codecov bot commented Jan 16, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
agentops/llms/providers/ollama.py 0.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@the-praxs
add auotgen integration tests
88d59f5
@the-praxs
change vcr config to not check body
b5165d1
@the-praxs
consolidate autogen integration tests into a single test
649500e
@the-praxs
update vcr config to handle dynamic body content in autogen integrati…
f98497f 
…on test
@the-praxs
re-record autogen cassette after new vcr config
1b09285
@the-praxs
Merge branch 'main' into fix/partners-integration-tests
fefb160
@the-praxs
Add code to redact jwt in response body
53129be
@the-praxs
re-record autogen cassette
f1e2932
@the-praxs
update vcr config to handle body response more robustly
4c81469
@the-praxs
re-record autogen cassette (again)
b7f761e
@the-praxs
Merge branch 'main' into fix/partners-integration-tests
403059d
@the-praxs
add taskweaver in provider integration tests and fix litellm integrat…
dd5ed08 
…ion test
@the-praxs
add taskweaver fixture to conftest
b5fc7cc
@the-praxs
fix ai21 version
86d5c6e
@the-praxs
ignore host huggingface.co
ef3a4d3
@the-praxs
add gemini provider integration test
42d9087
@the-praxs
modify model in groq provider integration test
7e8aecd
@the-praxs
re-record groq cassette
1c5f06a
@the-praxs the-praxs requested a review from teocns January 21, 2025 08:26
@the-praxs the-praxs marked this pull request as ready for review January 21, 2025 08:26
@the-praxs
Copy link
Member Author

Closed in favor of small PRs.

@the-praxs the-praxs closed this Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@teocns teocns Awaiting requested review from teocns

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@the-praxs @teocns