README.md

Vald

Overview

Vald is an open-source, distributed vector search engine for Kubernetes, enabling high-speed similarity searches on vectorized data using advanced algorithms like NGT.

Description

Vald 🚀is an open-source, cloud-native distributed vector search engine. It enables lightning-fast 🤯 similarity searches on billions of vectorized data points. Built for Kubernetes, it features horizontal scaling, customizable filtering, auto-indexing/backup, and uses advanced algorithms like NGT 🔎 under the hood.

💡 Vald Key Highlights

1. ✅ Distributed indexing across nodes for performance and stability

2. ✅ Auto-indexing with backups to handle failures gracefully

3. ✅ Custom ingress/egress filtering to manipulate data

4. ✅ Horizontal scaling on memory & CPU

5. ✅ Support for multiple languages via gRPC

Vald aims to make vector search feasible at scale for unstructured data like images, video, audio, text, etc. The goal is to help build next-gen search, recommendations 💡 and analysis using vector representations and approximate similarity.

As an OSS project, Vald is modular and hackable!

So, if you deal with vectors at scale and are building advanced search/recommendation systems, check out Vald!

🤔 Why should The AI Engineer care about Vald?

1. ⚡️ Enables lightning-fast vector similarity searches even on billions of vectors (unlike other libraries that get exponentially slower). It means quickly finding similar images, audio clips, documents, etc, without worrying about the scale.
2. 📈 Horizontally scalable on Kubernetes, you can easily add more memory/compute as your vector data grows. It saves you from re-architecting your search infrastructure every few months.
3. 💽 Auto-manages indexing and backups of vector indexes, freeing up engineering time to focus on building better models rather than worrying about failures or reliability.
4. ⚙️ Customizable ingress/egress filtering allows easy data manipulation like cleaning vectors, transforming dimensions, etc., before they get indexed or after results are retrieved. It reduces overall data processing time.
5. 🛠 Modular microservice architecture makes Vald easy to optimize, extend, and integrate into your environment. You can hack it to suit your specific needs.

📊 Tell me more about Vald!

• 👷🏽‍♀️ Builders: Yusuke Kato, Rintaro Okamura, Kiichiro YUKAWA, Hiroto Funakoshi
• 👩🏽‍💼 Builders on LinkedIn: https://www.linkedin.com/in/kpango/,
• 👩🏽‍🏭 Builders on X: https://twitter.com/kpang0, https://twitter.com/hiroto_hlts2
• 👩🏽‍💻 Contributors: 24
• 💫 GitHub Stars: 1.4k
• 🍴 Forks: 70
• 👁️ Watch: 20
• 🪪 License: Apache-2.0
• 🔗 Links: Below 👇🏽

🖇️ Where can I find out more about Vald?

• GitHub Repository: https://github.com/vdaas/vald
• Official Website: https://vald.vdaas.org/
• Slack Community: https://join.slack.com/t/vald-community/shared_invite/zt-db2ky9o4-R_9p2sVp8xRwztVa8gfnPA
• X Page: https://twitter.com/vdaas_vald
• Profile in The AI Engineer: https://github.com/theaiengineer/awesome-opensource-ai-engineering/blob/main/libraries/vald/README.md

---

🧙🏽 Follow The AI Engineer for more about Vald and daily insights tailored to AI engineers. Subscribe to our newsletter. We are the AI community for hackers!

♻️ Repost this to help Vald become more popular. Support AI Open-Source Libraries!

⚠️ If you want me to highlight your favorite AI library, open-source or not, please share it in the comments section!