Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/sqlite vec vector store #569

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

Heinrich-vanNieuwenhuizen
Copy link
Contributor

@Heinrich-vanNieuwenhuizen Heinrich-vanNieuwenhuizen commented Oct 10, 2024

Feature #516

Description

This pull request refactors and enhances the SQLite vector store implementation, transitioning from SQLiteVSS to SQLiteVec. The changes improve compatibility with Langchain 0.3 and introduce various optimizations and structural improvements.

Dependencies

  • Updated pubspec.yaml to reflect new dependencies and omissions.
  • Added a tool to download SQLite3 binaries, improving cross-platform compatibility.

Changes

  • Refactored SQLiteVSS to SQLiteVec, including file renames and code adjustments.
  • Updated database connection handling for native, web, and unsupported platforms.
  • Improved error handling and debugging utilities.
  • Restructured the vector store implementation for better maintainability and performance.

Testing

  • Updated and renamed test files to reflect the transition from SQLiteVSS to SQLiteVec.
  • Ensured test coverage for the refactored implementation.

Code Quality

  • Applied Dart formatting to ensure consistent code style.
  • Implemented Dart fixes to address potential issues and improve code quality.

Additional Notes

  • This PR includes changes across multiple commits, focusing on refactoring, compatibility updates, and structural improvements.
  • The transition to SQLiteVec aims to provide a more robust and efficient vector store implementation within the Langchain community package.

Happy with test contents and location
Solved Both source_gen:combining_builder and objectbox_generator:generator may output objectbox.g.dart
Next ensure clean implementation and then test
Need to add to melos script
@davidmigloz
Copy link
Owner

I started with the review 🙂
I need to familiarize a bit with sqlite3 and drift first.

@Heinrich-vanNieuwenhuizen
Copy link
Contributor Author

I started with the review 🙂 I need to familiarize a bit with sqlite3 and drift first.

No issues, let me know if you want to do a google meet up if you want me to take you through it.

@davidmigloz
Copy link
Owner

I've been sort of mental capacity the last few days. I'll resume the review tomorrow. I'll let you know if we need to jump in on a call, but thanks for the offer.

@Heinrich-vanNieuwenhuizen
Copy link
Contributor Author

I've been sort of mental capacity the last few days. I'll resume the review tomorrow. I'll let you know if we need to jump in on a call, but thanks for the offer.

anytime

@davidmigloz davidmigloz modified the milestones: 2024-10-29, 0.8.0 Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c:vector-stores Vector stores. p:langchain_community langchain_community package. t:enhancement New feature or request
Projects
Status: 🏗 In Progress
Development

Successfully merging this pull request may close these issues.

2 participants