Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sim search refactoring #466

Merged
merged 17 commits into from
Nov 20, 2024
Merged

sim search refactoring #466

merged 17 commits into from
Nov 20, 2024

Conversation

fynnos
Copy link
Collaborator

@fynnos fynnos commented Nov 13, 2024

This PR refactors the sim search service by splitting the functionality into multiple services:

Old: simsearch service is basically hard-wired to use weaviate

new: simsearch service provides generic methods to search for similar images, sentences, documents etc. it internally uses a vectorIndexService (via interface/abstract class)
to retain the old behavior, there is an implementation of the vectorindexService using weaviate. However, I've also included some partial implementations to try out typesense and qdrant instead.

Copy link

gitguardian bot commented Nov 13, 2024

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
11487172 Triggered Company Email Password 1a484f5 docker/.env.example View secret
11487172 Triggered Company Email Password d23c72c docker/.env.example View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@fynnos fynnos requested a review from bigabig November 14, 2024 11:04
@fynnos fynnos changed the title sim search refactoring for annoscaling sim search refactoring Nov 14, 2024
Copy link
Member

@bigabig bigabig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

puh. mit 3 vector dbs müssen einige neue configs in .env.example(s) eingeführt werden.

das depends_on in compose.yml wird ein echtes problem. bin gespannt ob das gelöst werden kann, ob man die irgendwie dynamisch machen kann

docker/compose.yml Show resolved Hide resolved
docker/compose.yml Outdated Show resolved Hide resolved
docker/compose.yml Outdated Show resolved Hide resolved
docker/compose.yml Show resolved Hide resolved
docker/.env.example Outdated Show resolved Hide resolved
.github/workflows/frontend_checks.yml Outdated Show resolved Hide resolved
backend/src/configs/production.yaml Outdated Show resolved Hide resolved
backend/src/configs/production.yaml Outdated Show resolved Hide resolved
backend/src/configs/production.yaml Outdated Show resolved Hide resolved
@bigabig bigabig merged commit f306212 into mwp_v1 Nov 20, 2024
4 checks passed
@bigabig bigabig deleted the feat/annoscaling branch November 20, 2024 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants