nlist set number

[wh336699]

i noticed that in ./condifs/milvus.yaml, the nlist set is
nlist: 128 # temp index nlist, recommend to set sqrt(chunkRows), must smaller than chunkRows/8
but numchunkRows set is chunkRows: 128 # Row count by which Segcore divides a segment into chunks.

In my understanding，sqrt(chunkRows) is not same as numchunkRows, first chunkRows means the all numbers of insert rows,and numchunkRows means how many chunk we should divide segment into.

Am I right?

[yhmo]

In each segment, data is stored as chunks, each chunk is a separate file in s3. Chunk is invisible to users, it is for managing the data in storage.
chunkRows: 128 means the minimum row count in each chunk. A segment contains multiple chunks so that the querynode/datanode no need to sync too much data from the message queue when the server is unexpectedly down.
Milvus automatically manages chunks, a chunk can be 128 rows, and another chunk can be 10000 rows, depending on one-row insertion or batch insertion. Small chunks can be merged into large chunks in a segment. Users no need to care about chunks.

nlist:128 is for interimIndex. Interim index is a temp index for quickly search in growing segment. So, nlist:128 is for a growing segment, not for a chunk.

Since the row number of a growing segment could continually increase, no need to use sqrt(segmentRows) to calculate the nlist. A const nlist value is ok.