fix(stream_family): Fix journaling in the XADD and XTRIM commands #4448
Conversation
| @@ -300,7 +300,7 @@ void streamGetEdgeID(stream *s, int first, int skip_tombstones, streamID *edge_i | |||
| * that should be trimmed, there is a chance we will still have entries with | |||
| * IDs < 'id' (or number of elements >= maxlen in case of MAXLEN). | |||
| */ | |||
| int64_t streamTrim(stream *s, streamAddTrimArgs *args) { | |||
There was a problem hiding this comment.
Can you please explain where is the randomness ?
If we run the same xadd command on master and replica (without your changes) how can it trim different number of elements?
There was a problem hiding this comment.
There is no randomness here. Internally, a stream is a rax tree of listpacks. When the user trims with the ~ option, it means that instead of trimming elements within the listpacks before the specified entry, only the rax tree nodes smaller than the specified entry are removed.
However, I still believe it’s worth adding this type of journaling, as the Redis API doesn’t guarantee anything in this regard.
Btw, most of the changes are related to fixing the options parsing for the XADD and XTRIM commands, so there isn’t a significant change in journaling
There was a problem hiding this comment.
If there is not randomness not sure what is exactly the bug that we have as replica should execute the exact same algorithm as master does. Can you explain how we can have a bug in replicaiton? @romange
There was a problem hiding this comment.
#4202 marked as "enhancement" so it's not a bug. While in theory there should not be any discrepancies, in practice I do not know if this can not happen due to a slightly different state of the underlying data structures or slight differences in clock. As I wrote in the issue, I prefer we do not replicate declarative commands but use commands that preserve the exact state if possible.
fixes dragonflydb#4202 Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
BagritsevichStepan
force-pushed
the
stream/fix-trim-journaling
branch
from
2ddf4cc to
f635b6a
Compare
fixes #4202