You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Ghostferry current ignores statement based replication events, which may cause data corruption if binlog_format and binlog_row_image is set temporarily to the wrong value
#183
Open
shuhaowu opened this issue
May 12, 2020
· 0 comments
binlog_format and binlog_row_image are session variables, which could cause certain data mutating events to be dropped by Ghostferry as it currently doesn't process events. An example program that alters binlog_format on its session is pt-heartbeat.
This is an "user-error" type problem. That said, the problem may cause data corruption that is invisible to the user, which is not acceptable. At the minimum, we need to update the documentations about such an occurrence near where we mention why binlog_format needs to be set to ROW. If users are running with the InlineVerifier, there's a chance that these events will be caught by the verifier, but I don't think it is guaranteed.
Ideally, Ghostferry should know this is wrong and error out if it observes such an event. That said, we may need to exclude some SBR entries like pt-heartbeat, or tell users to ignore those schemas/tables all together. These problems could be tricky to resolve. That said, throwing an error due to any of these events would be a cautious first step.
The text was updated successfully, but these errors were encountered:
binlog_format
andbinlog_row_image
are session variables, which could cause certain data mutating events to be dropped by Ghostferry as it currently doesn't process events. An example program that altersbinlog_format
on its session is pt-heartbeat.This is an "user-error" type problem. That said, the problem may cause data corruption that is invisible to the user, which is not acceptable. At the minimum, we need to update the documentations about such an occurrence near where we mention why binlog_format needs to be set to ROW. If users are running with the InlineVerifier, there's a chance that these events will be caught by the verifier, but I don't think it is guaranteed.
Ideally, Ghostferry should know this is wrong and error out if it observes such an event. That said, we may need to exclude some SBR entries like pt-heartbeat, or tell users to ignore those schemas/tables all together. These problems could be tricky to resolve. That said, throwing an error due to any of these events would be a cautious first step.
The text was updated successfully, but these errors were encountered: