[Feature] defined mongo cdc schema on table_conf

[bulolo]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

according to this issue #4683

we find that mongo cdc change schema frequently，and can not query right value on starrock paimon cagalog

Solution

we can defined a schema by set permanent type and filed to save it

like seatunnel we can defined a schema on mongo cdc table_conf

<FLINK_HOME>/bin/flink run \
    /path/to/paimon-flink-action-0.9.0.jar \
    mongodb_sync_table \
    --warehouse hdfs:///path/to/warehouse \
    --database test_db \
    --table test_table \
    --partition_keys pt \
    --computed_column '_year=year(age)' \
    --mongodb_conf hosts=127.0.0.1:27017 \
    --mongodb_conf username=root \
    --mongodb_conf password=123456 \
    --mongodb_conf database=source_db \
    --mongodb_conf collection=source_table1 \
    --catalog_conf metastore=hive \
    --catalog_conf uri=thrift://hive-metastore:9083 \
    --table_conf bucket=4 \
    --table_conf changelog-producer=input \
    --table_conf sink.parallelism=4 \ 
    --table_conf schema={
      fields {
        "_id" : int,
        "name" : string,
        "created" : datetime,
        "isDelete" : boolean
      }
    }

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!