Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore the MVCC functionality. #10

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open

Restore the MVCC functionality. #10

wants to merge 20 commits into from

Conversation

czs007
Copy link
Owner

@czs007 czs007 commented Jan 5, 2024

When the TimeTravel functionality was previously removed, it inadvertently affected the MVCC functionality within the system. This PR aims to reintroduce the internal MVCC functionality as follows:

  1. Add MvccTimestamp to the requests of Search/Query and the results of Search internally.
  2. When the delegator receives a Query/Search request and there is no MVCC timestamp set in the request, set the delegator's current tsafe as the MVCC timestamp of the request. If the request already has an MVCC timestamp, do not modify it.
  3. When the Proxy handles Search and triggers the second phase ReQuery, divide the ReQuery into different shards and pass the MVCC timestamp to the corresponding Query requests.

@czs007 czs007 force-pushed the recover_mvcc branch 2 times, most recently from f1ff880 to f69e798 Compare January 5, 2024 08:54
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Copy link

@longjiquan longjiquan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

bigsheeper and others added 13 commits January 5, 2024 18:12
This PR implements a new json reader for import.

issue: milvus-io#28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
fix milvus-io#29642

Signed-off-by: yah01 <yang.cen@zilliz.com>
Resolves: milvus-io#29736

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: milvus-io#29697

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
This PR implements a Parquet reader for import.

issue: milvus-io#28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
milvus-io#28410
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
…#29627)

related to : milvus-io#29417 

cardinal indexes upload index files in `Serialize` interface, and throw
exception when the `Serialize` failed.

Signed-off-by: xianliang <xianliang.li@zilliz.com>
fix: milvus-io#29585

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
fix: milvus-io#29349

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
milvus-io#26137

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
This pr will make milvus load delta logs concurrently, which should
decrease the latency of loading a segment.
/kind improvement

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
issue: milvus-io#29654

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
congqixia and others added 5 commits January 8, 2024 14:16
…ilvus-io#29740)

See also milvus-io#27349

The segment level label in querynode used `Legacy` before segment level
was correctly passed in Load request. Now this attribute is still using
legacy so the metrics does not look right.

This PR add paramter for `NewSegment` and passes corrent values for each
invocation.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
…9747)

Once a role is granted to a user, the user should automatically possess
the privilege information associated with that role.

issue: milvus-io#29710

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
issue milvus-io#25639 

Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
this avoids the conflicts while running multiple unit tests

Signed-off-by: yah01 <yah2er0ne@outlook.com>
When the TimeTravel functionality was previously removed, it inadvertently
affected the MVCC functionality within the system.
This PR aims to reintroduce the internal MVCC functionality as follows:

 1. Add MvccTimestamp to the requests of Search/Query and the results of Search internally.
 2. When the delegator receives a Query/Search request and there is no MVCC timestamp set
    in the request, set the delegator's current tsafe as the MVCC timestamp of the request.
    If the request already has an MVCC timestamp, do not modify it.
 3. When the Proxy handles Search and triggers the second phase ReQuery, divide the ReQuery
    into different shards and pass the MVCC timestamp to the corresponding Query requests.

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.