Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-37000] Fix MySQL CDC could not handle datetime prior to 1970 properly #3834

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

yuxiqian
Copy link
Contributor

@yuxiqian yuxiqian commented Jan 3, 2025

Originally reported by @WuChongYong.


Currently, MySQL connector couldn't properly handle Date and DateTime records prior to 1970-01-01 (Unix epoch).

  • DateTime records in Year 0000 are always null.

This behavior is actually expected, since 0000 A.D. does not exist. Common Era starts at Year 0001.

This could not be easily fixed since lots of libraries including MySQL connector and Java Calendar does not really support Year 0000 well (will be converted to 0001 automatically).

  • DateTime Records between Year 0001 and 0099 are Shifted to 1970 and 2069.

This could be overridden by debezium.enable.time.adjuster option.

  • DateTime Records between 0100 and 1969 with precision digits are not serializable.

This is caused by a glitch in RowDataDebeziumDeserializeSchema and DebeziumEventDeserializationSchema.


Most part of codes are just tests to cover this rarely used scenario.

…roperly

Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant