Why can't I connect remote Milvus server in Python SDK?

[lingvisa]

A remote Milvus distributed instance is deployed. And I can access it through "curl":

 curl --request GET \
    --url 'https://rag-db.c-97a1353.stage.kyma.ondemand.com/v1/vector/collections' \
    --header "Authorization: Bearer xxx:xxx" \
    --header 'accept: application/json' \
    --header 'content-type: application/json'
{"code":200,"data":["demo"]}%    

But then in my Python client, it reports this error:

from pymilvus import MilvusClient
client = MilvusClient(uri="https://rag-db.c-97a1353.stage.kyma.ondemand.com", token="xxx:xxx", db_name="default")
print(client)

The error message is:

raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
   status = StatusCode.UNAUTHENTICATED
   details = "Received http2 header with status: 401"
   debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"Received http2 header with status: 401", grpc_status:16, created_time:"2024-09-12T17:25:47.431059-07:00"}"

In both cases, the credentials are the same: "xxx:xxx". My version:
pymilvus==2.4.6

Why is that?

[lingvisa]

Hello, this is an urgent issue. Can you please help me with this? Thank you

[yhmo]

This is not a problem of milvus/pymilvus. Search the keyword "Received http2 header with status: 401" to get more info.

[lingvisa]

Authentification turned off for the time being. I loaded some documents, one collection with multiple partitions. But why does it show each individual

's partition's count is 0, and all the counts are under 'default partition'?

[yhmo]

The interface to get row count of partitions is a pretty old interface, the returned number is not real-time.
If you click the "Flush" button in the "Segments" page, you will see the row count updated.

[lingvisa]

So I have 8 partitions and they are placed into two segments.
Another little issue is, the function 'collection.has_index()' only works for a single index. If I have two indexes, this function reports an error, saying that it has multiple indexes, so it's ambiguous. I can use 'collection.indexes' to test whether it has any index as a workaround.

[yhmo]

has_index() has a parameter 'index_name". By default, index name is the vector field name.

import random

from pymilvus import (
    FieldSchema, CollectionSchema, DataType,
    utility, Collection, connections
)

connections.connect(host='localhost',port='19530')

schema = CollectionSchema(fields=[
    FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=False),
    FieldSchema(name="vector1", dtype=DataType.FLOAT_VECTOR, dim=128),
    FieldSchema(name="vector2", dtype=DataType.FLOAT_VECTOR, dim=256),
])

collection_name = "BBB"
utility.drop_collection(collection_name)
collection = Collection(collection_name, schema)

index_params_ivf={"index_type": "IVF_FLAT", "params": {"nlist": 128}, "metric_type": "L2"}
collection.create_index("vector1", index_params=index_params_ivf)
collection.create_index('vector2', index_params=index_params_ivf)

for idx in collection.indexes:
    print(idx.to_dict())

print("vector1 has index:", collection.has_index(index_name="vector1"))
print("vector2 has index:", collection.has_index(index_name="vector2"))

collection.drop_index(index_name="vector1")
print("vector1 has index:", collection.has_index(index_name="vector1"))

[lingvisa]

Right. I should use "index_name=xxx".

[lingvisa]

@yhmo Is it possible that langchain can also provide a version of MilvusCollectionHybridSearchRetriever that can return the scores as the non-hybrid version does as below?

retriever = Milvus(
        dense_embedding_func,
        collection_name=collection_name,
        vector_field="dense_vector",
        consistency_level="Strong",
    )

Although I can create multiple partitions in one collection in Milvus in hybrid search, that also requires to create a single keyword index for the whole document set. If I have documents from different sources it is logical to create one collection for each data source, and this will reduce the bm25 index building's scalability requirement.

Does this make sense, or any recommendation on this concern?

[lingvisa]

When do i need to use flush() and load() in creating a new collection?

           collection = Collection(self.collection_name, schema, consistency_level="Strong") 
           collection.create_partition(partition_name=self.partition_name) 
           sparse_index = {"index_type": "SPARSE_INVERTED_INDEX", "metric_type": "IP"}
           collection.create_index("sparse_vector", sparse_index)
           dense_index = {"index_type": "AUTOINDEX", "metric_type": "IP"}
            collection.create_index("dense_vector", dense_index)

Now, do i need the two statements below:

collection.flush()
collection.load()

Or are they only needed when I insert new entities to the collection?

 for i in range(0, len(docs), batch_size):
            batched_entities = [
                docs[i: i + batch_size],
                sparse_embeddings[i: i + batch_size],
                dense_embeddings[i: i + batch_size],
            ]
        collection.insert(batched_entities)
        collection.flush()
        collection.load()

[yhmo]

You don't need to call flush(). In fact, we strongly recommend users don't call flush() interface manually. Milvus internally automatically flushes data.

The following workflow is good practice:
create_collection() ----> insert() ---> ..... ---> insert() ---> create_index() ----> load() ---> search()---> ... ---> search()

[lingvisa]

why does create_index() run after insert()? create_index() should occur when a collection is initialized or created, and then you insert() data, right?

[xiaofan-luan]

why does create_index() run after insert()? create_index() should occur when a collection is initialized or created, and then you insert() data, right?

you can do both.
either create index first or insert data first.
Personally I recommend to create index first then insert data

[lingvisa]

That's good to know! @xiaofan-luan

[lingvisa]

@xiaofan-luan and @yhmo For MilvusCollectionHybridSearchRetriever,

 retriever = MilvusCollectionHybridSearchRetriever(
        collection=collection,
        # rerank=WeightedRanker(0.5, 0.5),
        rerank=RRFRanker(),
        anns_fields=["dense_vector", "sparse_vector"],
        field_embeddings=[dense_embedding_func, sparse_embedding_func],
        field_search_params=[dense_search_params, sparse_search_params],
        top_k=TOP_K,
        text_field='text',
    )

If I change top_k from 5 to 20 without changing anything else, the orders of ranked results are not exactly the same. No matter I use 5 or 20, the top 5 results should be exactly the same, right?

[yhmo]

Definitely different.
The topk value will affect hybrid_search result. Read this discussion for more info: #36329

[lingvisa]

In the example,

connections.connect(
    host="localhost",
    port="19530",
    user="root",
    password="password",
    db_name="default",
)
print(utility.get_server_version())
collection = Collection(collection_name)
client = MilvusClient(
    host="localhost",
    port="19530",
    user="root",
    password="password",
    db_name="default",
)

If I have the collection object, I don't need another 'client' object. The 'collection' works for both hybrid and non-hybrid search, right? is the MilvusClient() the old API, and "collections.connect(...), collection=Collection(collection_name)" are the new API?

[lingvisa]

Once a collection is created and I want to switch to another similarity metric for testing, can I switch similarity metric in web-based browser, or I need to do it in code without the need to re-create the collection?

Also, how to test searches in the browser? Please see the attachment.

[lingvisa]

The consistency_level="Bounded" does improve:

2024-11-12 22:24:30 - INFO - build_embedding_index.py:107 - Time per query: 0.17090392112731934 seconds
2024-11-12 22:24:30 - INFO - build_embedding_index.py:107 - Time per query: 0.15651202201843262 seconds
2024-11-12 22:24:30 - INFO - build_embedding_index.py:107 - Time per query: 0.16710615158081055 seconds
2024-11-12 22:24:31 - INFO - build_embedding_index.py:107 - Time per query: 0.13741731643676758 seconds
2024-11-12 22:24:31 - INFO - build_embedding_index.py:107 - Time per query: 0.13189983367919922 seconds
2024-11-12 22:24:31 - INFO - build_embedding_index.py:107 - Time per query: 0.19162392616271973 seconds

But still much slower than yours. Is it due to the sparse_vector I have, which you don't have it?

[lingvisa]

Another diff is that I have "Strong" in schema definition:
self.collection = Collection(self.collection_name, schema, consistency_level="Strong")

[yhmo]

"Strong" in schema definition is the default level.
If you didn't specify consistency_level in query(), it will use the default level.

Maybe your milvus server is remote, the result is transferred from network with poor bandwidth. My milvus is on local machine.

[lingvisa]

Yes. that's possible. So here the consistency_level in query() function overwrites the schema definition, which can speed up. If I want to optimize latency through consistency_leve, can I do it at query time? In langchain:

chunks = retriever.similarity_search(query, k=10)

Where retriever is a Milvus client. Can I change consistency_level here? Or I need to change it in schema definition.

[yhmo]

similarity_search() internally calls the collection.search(), the extra parameters are passed by the "kwargs". So, you can set consistency_level in similarity_search().

[lingvisa]

Hi, @yhmo

async embed_and_insert_documents(self, ...):

    ...

    self.collection.insert(batched_entities, partition_name=partition_name)

Do I need to use a lock for the insert() function when adding entities into milvus in an async context? Or internally it already takes care of concurrency issue so I don' t need to worry about it?

[yhmo]

No need to add lock, milvus insert() interface supports parallel insertion.