Vote if you need a local Milvus that runs in client library

[codingjaguar]

There are many community members expressed a need of lightweight Milvus, something that can be started from a client library. If you don't have millions of vectors and just want something quick and easy, what if there is:

$ pip install pymilvus
client = MilvusClient("~/my_milvus.db")
client.create_collection('medium_articles', ...)

No need of docker, API key, endpoints. You can start ingest and search vectors right away!

Even better, you might be able to ingest text directly, and it will convert to vector for you, e.g.

client.insert(collection_name, [
        {"id": 1, "vec_from": "interprets the meaning of words and phrases", "price": 10},
        {"id": 2, "vec_from": "analyze a piece of text to determine the sentiment behind it", "price": 20},
])
client.search(collection_name, data = ["what's semantic search"])

The local version of Milvus will fit in client library, be even more lightweight than https://github.com/milvus-io/milvus-lite and will replace it.

Let us know how you think of this idea! 🙋

[fgreco55]

For many IoT use cases, an embedded Milvus would be tremendously helpful. Also, if an embedded Milvus can act as a local cache to an external Milvus server, that would awesome.

[codingjaguar]

That's an interesting perspective! For IoT case, is there any preference on language? Is python or JavaScript popular in that scenario? The caching is an interesting idea, can you elaborate on the use case? Is that also for IoT? Does it need the server side to push data to the terminal nodes?

[fgreco55]

Unfortunately there's no real (mid-to-high) language standard for embedded other than C. My particular use case is primarily with Java but it's polyglot with collaborating AI/ML entities. Modern Java allows me to create a (smallish) native, secure executable but less-endowed endpoints may not be Java. They most likely will be C or Python. The nodes will all communicate and collaborate, so the major nodes will prob be the ones with the AI/ML intelligence.

Caching is for another use case. In enterprise apps (esp mobile ones), caching of traditional SQL databases has been done for a long time. It's valuable for performance as well as security/auditing. There are various levels of DB caching from the hardware to the endpoint. Applying this pattern to VDBs for IoT would be a natural fit. Edge AI is a humongous use case. And AI needa a VDB.

[xiaofan-luan]

Fully agreed a lite version of milvus running on small device is a interesting use case.
Under these use cases, c/cpp is still the main stream language. Python and go might be useful as well

[christy]

Adding stackoverflow comment here:
https://stackoverflow.com/questions/77486580/is-pymilvus-client-thread-safe-fork-safe

I'm thinking about using Milvus vector storage in my Flask based project and looking at the PyMilvus (Python SDK) documentation. I haven't found any information yet about:

Is PyMilvus thread-safe?
Is PyMilvus fork-safe?
How does connection pooling work in the SDK?
Could you help me to sort it out?

The official documentation doesn't contain too much information.

[yhmo]

Currently PyMilvus version(v2.3.x) doesn't provide a thread pool or connection pool. Basically, PyMilvus has a global object "connections" to maintain client-to-server connections.

User calls connections.connect() To create a connection:

from pymilvus import (
    connections,
}
connections.connect(host=HOST, port=PORT, alias="xxx")

This method has a parameter "alias", it is the name of the connection.
The "connections" object internally maintains a map of name-to-connection. If you didn't provide the "alias", it will use "default" as the name of the connection.

When you declare a collection, there is a parameter "using" to specify a connection name. If you didn't provide the "using", it will use "default" connection. All the interfaces of this Collection will via this connection.

collection = Collection(name=collection_name, using="xxx")
collection.insert()
collection.search()
......

The connection object is thread-safe, which means you can call the collection's interfaces from different threads.
But the connection object cannot be shared by multiple sub-processes. So, if you fork a sub-process, you should ensure each subprocess creates its own connection by connections.connect().