2.3.8 Release

2.3.8 Release Note

Bug fix

Core

• [Hotfix][Core] Fix concurrency exceptions when MultiTableSink#PrepareCommit (#7686)
• [hotfix][zeta] conf property is null, print log bug fix (#7487)
• [Fix][Seatunnel-core]Fix syntax error in the execution script on Windows (#7423)

Connectors

• [Fix][Connecotr-V2] Fix paimon dynamic bucket tale in primary key is not first (#7728)
• [Fix][Connector-V2] Fix iceberg throw java: package sun.security.krb5 does not exist when use jdk 11 (#7734)
• [Fix][Connector-V2] Release resources when task is closed for iceberg sinkwriter (#7729)
• [Fix][Connector-V2] Release resources even the task is crashed for paimon sink (#7726)
• [Fix][Connector-V2] Fix paimon e2e error (#7721)
• [Fix][Connector-V2] Fix known directory create and delete ignore issues (#7700)
• [Fix][Connector-V2] Fixed iceberg sink can not handle uppercase fields (#7660)
• [Hotfix][CDC] Fix ddl duplicate execution error when config multi_table_sink_replica (#7634)
• [Fix][Connector-V2] Fix some throwable error not be caught (#7657)
• [Fix][Connector-V2] Fix jdbc test case failed (#7690)
• [Fix][Connector-v2] Throw Exception in sql query for JdbcCatalog in table or db exists query (#7651)
• [Fix][JDBC] Fix starrocks jdbc dialect catalog conflict with starrocks connector (#7578)
• [bugfix] fix oracle query table length (#7627)
• [Hotfix] Fix iceberg missing column comment when savemode create table (#7608)
• [Hotfix][Seatunnel-common] Fix the CommonError msg for paimon sink (#7591)
• [Hotfix][Connector-v2] Fix the ClassCastException for connector-mongodb (#7586)
• [Hotfix][Connector-v2] Fix the NullPointerException for jdbc oracle which used the table_list (#7544)
• [Bug][e2e][jdbc-gbase] The gbase driver address certificate expired (#7531)
• [Hotfix][Connector-V2] Fixed lost data precision for decimal data types (#7527)
• [Fix][Connector-V2] Fix jdbc query sql can not get table path (#7484)
• [Hotfix][Connector-V2] Fix null not inserted in es (#7493)
• [Fixbug] doris custom sql work (#7464)
• [Fix] Fix oracle sample data from column error (#7340)
• [Hotifx][Jdbc] Fix MySQL unsupport 'ZEROFILL' column type (#7407)
• [Hotfix][CDC] Fix package name spelling mistake (#7415)
• [Hotfix][Zeta] Fix hazelcast client should overwrite cluster address by system env (#7790)

Zeta(ST-Engine)

• [Core][Zeta] If Zeta not a TCP discovery, it cannot find other members (#7757)
• [Fix][Zeta] Fix resource isolation not working on multi node (#7471)

Transformer

• [Bug][Transforms-V2] Fix LLM transform can not parse boolean value type (#7620)

CI

• [Fix][CI] Fix CI loss document check when raise PR (#7749)
• [Hotfix][ci] Fix github ci License header error (#7738)
• [Fix][CI] Add doc sync to fix ci error (#7720)
• [hotfix] fix FixSlotResourceTest unstable issue (#7577)
• [Fix][e2e] remote loading driver ignores the certificate to avoid certificate address expiration (#7547)
• [Fix] Fix document build error (#7546)
• [Fix][doc] fix dead link (#7508)
• [Fix] update paimon.apache.org deadlink (#7504)
• [Hotfix][Metrics] fix sporadic multi-metrics ci (#7468)
• [Fix] Fix dead link on seatunnel connectors list url (#7453)
• [FIX][E2E]Modify the OceanBase test case to the latest imageChange image (#7452)

Improve

Core

• [Improve][Spark] Convert array type to exact type (#7758)
• [Improve][Zeta] Split the classloader of task group (#7580)
• [Improve][Core] Config variables update doc and add test case (#7709)
• [Improve][Zeta] Add log for tryTriggerPendingCheckpoint because the wrong time of server (#7717)
• [Improve][EventService] improve event code and extract event code to EventService (#7153)
• [Chore] Code specification adjustments (#7572)
• [Chore] Update zeta example log4j2.properties (#7563)
• [Improve] Update docker doc and build tag (#7486)
• [Improve][Zeta] Handle user privacy when submitting a task print config logs (#7247)
• [Improve][API] Add IGNORE savemode type (#7443)
• [Improve][API] Move catalog open to SaveModeHandler (#7439)
• [Improve] Skip downloading transitive dependency when install plugins (#7374)
• [Improve] Flink support embedding transform (#7592)

Connector-V2

• [Improve][Connector-v2] Remove useless code and add changelog doc for paimon sink (#7748)
• [Improve][Connector-V2] Optimize sqlserver package structure (#7715)
• [Improve][Connector-V2] Optimize milvus code (#7691)
• [Improve][Redis]Redis scan command supports versions 5, 6, 7 (#7666)
• [Improve][Connector-V2] Support read archive compress file (#7633)
• [Improve][Jdbc] Jdbc truncate table should check table not database (#7654)
• [Improve] [Connector-V2] Optimize milvus-connector config code (#7658)
• [Improve][Connector-V2] Time supports default value (#7639)
• [Improve][Iceberg] Add savemode create table primaryKey testcase (#7641)
• [Improve][Kafka] kafka source refactored some reader read logic (#6408)
• [Improve][Connector-V2][MongoDB] A BsonInt32 will be convert to a long type (#7567)
• [Improve][Connector-v2] Improve the exception msg in case-sensitive case for paimon sink (#7549)
• [Improve][Connector-v2] Support mysql 8.1/8.2/8.3 for jdbc (#7530)
• [Improve][Connector-v2] Release resource in closeStatements even exception occurred in executeBatch (#7533)
• [Improve][Connector-V2] Remove hard code iceberg table format version (#7500)
• [Improve][Connector-V2] Fake supports column configuration (#7503)
• [Improve][Connector-V2] update vectorType (#7446)
• [Improve] Improve some connectors prepare check error message (#7465)
• [Improve] Refactor S3FileCatalog and it's factory (#7457)
• [Improve] Added OSSFileCatalog and it's factory (#7458)
• [Improve][Connector-V2] Reuse connection in StarRocksCatalog (#7342)
• [Improve][Connector-V2] Remove system table limit (#7391)
• [Improve][Connector-V2]Support multi-table sink feature for email (#7368)
• [Improve] Update pull request template and github action guide (#7376)
• [Improve][Connector-V2] Close all ResultSet after used (#7389)
• [Improvement] add starrocks jdbc dialect (#7294)

Transform

• [Improve][Transform] Improve inner sql query for map field type (#7718)
• [Improve][Transform] Support errorHandleWay on jsonpath transform (#7524)
• [Improve][Transform-v2] Refactor a transformRow from FilterFieldTransform (#7598)
• [Improve][Transform] Add LLM model provider microsoft #7778

CI

• [Improve][CI] Move paimon into single task (#7719)
• [Improve][Test][Connector-V2][MongoDB] Add few test cases for BsonToRowDataConverters (#7579)
• [Improve][Test] Remove useless code of S3Utils. (#7515)
• [Improve][E2E] update doris image to official version #7773

Feature

Core

• [Feature][Flink] Support multiple tables read and write (#7713)
• [Feature][Zeta][Core] Support output log file of job (#7712)
• [Feature][REST-API] Add threaddump rest api (#7615)
• [Feature][rest-api] Add whether master node identifier (#7603)
• [Feature][Core] shell batch cancel task (#7612)
• [Feature] Support config variable substitution with default value (#7562)
• [Feature][zeta]Support exposing monitoring metrics by prometheus exporter protocol (#7564)
• [Feature][Zeta] add rest api to update node tags (#7542)
• [Feature][Zeta] Optimized llm doc && add DOUBAO LLM (#7584)
• [Feature][Zeta] Support slf4j mdc tracing job id output into logs (#7491)
• [Feature][Core] Added rest-api for batch start and stop (#7522)
• [Feature][Core] Add event notify for all connector (#7501)
• [Feature] add dockerfile (#7346)
• [Feature][Zeta] Added other metrics info of multi-table (#7338)
• [Feature][Spark] Support multiple tables read and write (#7283)

Connector-V2

• [Feature][Connector-V2] Assert support multi-table check (#7687)
• [Feature][Connector-Paimon] Support dynamic bucket splitting improves Paimon writing efficiency (#7335)
• [Feature][Connector-v2] Support streaming read for paimon (#7681)
• [Feature][Connector-V2] Optimize hudi sink (#7662)
• [Feature][Connector-V2] jdbc saphana source tablepath support view and synonym (#7670)
• [Feature][Connector-V2] Ftp file sink suport multiple table and save mode (#7665)
• [Feature] Support tidb cdc connector source #7199 (#7477)
• [Feature][kafka] Add arg poll.timeout for interval poll messages (#7606)
• [Feature][Connector-V2][Hbase] implement hbase catalog (#7516)
• [Feature][Elastic search] Support multi-table source feature (#7502)
• [Feature][CONNECTORS-V2-Paimon] Paimon Sink supported truncate table (#7560)
• [Feature][Connector-V2] Support Qdrant sink and source connector (#7299)
• [Feature][Connector-V2] Support multi-table sink feature for HBase (#7169)
• [Feature][Connector-V2] Support typesense connector (#7450)
• [Feature][Rabbitmq] Allow configuration of queue durability and deletion policy (#7365)
• [Feature][Connector-V2] Add decimal_type_narrowing option in jdbc (#7461)
• [Feature][connector-v2]Add Kafka Protobuf Data Parsing Support (#7361)
• [Feature][Connector-V2][Tablestore] Support Source connector for Tablestore #7448 (#7467)
• [Feature][Connector-V2] Support opengauss-cdc (#7433)
• [Feature][Connector-V2] Suport choose the start page in http paging (#7180)
• [Feature][Connector-V2][OceanBase] Support vector types on OceanBase (#7375)
• [Feature][Connector-V2] Fake Source support produce vector data (#7401)
• [Feature][Connector-V2][Iceberg] Support Iceberg Kerberos (#7246)
• [Feature][Connector-V2] SqlServer support user-defined type (#7706)
• [Feature][Connector-V2] sftp file sink suport multiple table and save mode (#7668)

Transform

• [Feature][Transforms-V2] LLM transforms Support custom field name (#7640)
• [Feature][Transforms-V2] LLM transforms Support KimiAI (#7630)
• [Future][Transforms-V2] llm trans support field projection (#7621)
• [Feature][Transform] Add embedding transform (#7534)

Docs

• [Docs] Spark use doc update (...

2.3.7 Release

Bug fix

Connector-V2

• [Hotfix][MySQL-CDC] Fix ArrayIndexOutOfBoundsException in mysql binlog read (#7381)
• [Fix][Doris] Fix doris primary key order and fields order are inconsistent (#7377)
• [Bugfix][Doris] Fix Json serialization, null value causes data error problem (#7379)
• [Hotfix][Jdbc] Fix jdbc compile error (#7359)
• [Fix][OceanBase] Remove OceanBase catalog's dependency on mysql driver (#7311)
• [Fix][Tdengine] Fix sql exception and concurrentmodifyexception when connect to taos and read data (#6088)
• [Hotfix][Kafka] Fix kafka consumer log next startup offset (#7312)
• [Fix][Doris] Fix the abnormality of deleting data in CDC scenario. (#7315)
• [hotfix][Hbase]fix and optimize hbase source problem (#7148)
• [Fix][Iceberg] Unable to create a source for identifier 'Iceberg'. #7182 (#7279)

Zeta(ST-Engine)

• [Fix][Zeta] Fix task can not end cause by lock metrics failed (#7357)
• [Hotfix][Zeta] Fix task cannot be stopped when system is busy (#7292)
• [Hotfix][Zeta] Fix task cannot be stopped when system is busy (#7280)

E2E

• [Fix][Http] Fix http e2e case (#7356)

Improve

Core

• [Improve][Flink] optimize method name (#7372)
• [Improve][API] Check catalog table fields name legal before send to downstream (#7358)
• [Improve][Flink] refactor flink proxy source/sink (#7355)
• [Improve][API] Make sure the table name in TablePath not be null (#7252)
• [Improve][Core] Improve base on plugin name of lookup strategy (#7278)

Connector-V2

• [Improve][multi-table] Add multi-table sink option check (#7360)
• [Improve][Console] Update ConsoleSinkFactory.java (#7350)
• [Improve][Jdbc] Skip all index when auto create table to improve performance of write (#7288)
• [Improve][Doris] Improve doris error msg (#7343)
• [Improve][Jdbc] Remove MysqlType references in JdbcDialect (#7333)
• [Improve][Jdbc] Merge user config primary key when create table (#7313)
• [Improve][Jdbc] Optimize the way of databases and tables are checked for existence (#7261)

Transforms-V2

• [Improve][DynamicCompile] Improve DynamicCompile transform (#7319)
• [Improve][SQL] Remove escape identifier from output fields (#7297)
• [Improve][DynamicCompile] Improve DynamicCompile transform (#7264)

E2E

• [Improve][Improve] Enable fakesource e2e of spark/flink (#7325)
• [Improve][Improve] Enable JdbcPostgresIdentifierIT (#7326)
• [Improve][Improve] Support windows for the e2e of paimon (#7329)

Feature

Connector-V2

• [Feature][Aliyun SLS] add Aliyun SLS connector #3733 (#7348)
• [Feature][Activemq] Added activemq sink (#7251)

Transforms-V2

• [Feature] Split transform and move jar into connectors directory (#7218)
• [Feature][LLM] Add LLM transform (#7303)
• [Feature][SQL] Support cast to bytes function of sql (#7284)

Docs

• [Docs] Change deprecated connector name in setup.md (#7366)
• [Docs] Fix username parameter error in sftp sink document (#7334)
• [Docs] fix document configuration is rectified when the oss is selected as the checkpoint base (#7332)
• [Docs] Fix miss sink-options-placeholders.md in sidebars (#7310)
• [Docs] Update Oracle-CDC.md (#7285)
• [Docs] Fix hybrid cluster deployment document display error (#7306)
• [Docs] translate event-listener doc into chinese (#7274)

Others

• Bump org.apache.activemq:activemq-client (#7323)
• [Improve] Remove unused code (#7324)
• [Improve] Update snapshot version to 2.3.7 (#7305)

2.3.6 Release

2.3.6 Release Note

Bug fix

Core

• [Fix][Flink]Hotfix flink checkpoint default value (#7040)
• [Hotfix][Core] Fix the NullPointException for the json config of the job without pluginname (#6815)
• [Fix][Dist] Remove aws jar dependency in release package (#7257)

Connectors

• [Fix][Connector-V2] The isNullable attribute is true when the primary key field in the Paimon table converts the Column object. #7231 (#7242)
• [Fix][Connector kafka]Fix Kafka consumer stop fetching after TM node restarted (#7233)
• [Fix] Fix Hana type converter decimal scale is 0 convert to int error (#7167)
• [BugFix][Connector-V2][Maxcompute]fix:Maxcompute sink can't map field(#7164) (#7168)
• [Fix][Connector-V2] Fix kafka batch mode can not read all message (#7135)
• [Fix][Connector-V2][Elasticsearch]Fix sink configuration for DROP_DATA (#7124)
• [Fix][Connector-V2][CDC] SeaTunnelRowDebeziumDeserializationConverters NPE (#7119)
• [Hotfix][Connector-V2] fix rabbit checkpoint exception in Flink mode (#7108)
• [Fix][Connector-V2] Fix starrocks Content-Length header already present error (#7034)
• [Hotfix][MySQL-CDC] Fix read gbk varchar chinese garbled characters (#7046)
• [Hotfix][Hive Connector] Fix Hive hdfs-site.xml and hive-site.xml not be load error (#7069)
• [Fix] hive load hive_site_path and hdfs_site_path too late (#7017)
• [Fix] fix pgsql create table comment special string bug (#7022)
• [Fix] Fix plugin_config (#7104)
• [connector-jdbc][bugfix] fix sqlServer create table comment special string bug (#7024)
• [connector-jdbc][bugfix] fix oracle create table comment special string bug (#7012)
• [Hotfix][CDC] Fix split schema change stream (#7003)
• [bugfix] fix mysql create table comment special string bug (#6998)
• [Paimon]support projection for paimon source (#6343)
• [bugfix][connector-mongodb] fix mongodb null value write (#6967)
• [Hotfix][Connector-V2][kafka]Kafka consumer group automatically commits offset logic error fix (#6961)
• [Bug] [connector-hive] Eanble login with kerberos for hive (#6893)
• [Bug][Connector-Iceberg]fix create iceberg v2 table with pks (#6895)
• [Fix] correct the typo of kudu kerberos config (#6905)
• [Hotfix][Connector-V2] Close the tableWrite when task is close (#6897)
• [Fix][Connector-V2] Field information lost during Paimon DataType and SeaTunnel Column conversion (#6767)
• [Hotfix][Connector-V2] Fix the batch write with paimon (#6865)
• [Hotfix][Jdbc] Fix oracle savemode create table (#6651)
• [bugFix][Connector-V2][Doris] The multi-FE configuration is supported (#6341)
• [Hotfix][Postgres-CDC/OpenGauss-CDC] Fix read data missing when restore (#6785)
• [Fix] Fix HttpSource bug (#6824)
• [Hotfix][Jdbc/CDC] Fix postgresql uuid type in jdbc read (#6684)
• [Hotfix][Dist] Add oracle cdc (#6818)
• [Fix] fix Debezium conf "key_convert_schemas.enable" set false lead to nullPointException (#6809)
• [Fix][MySQL]: Fix MySqlTypeConverter could not be instantiated (#6781)
• [Fix][Connector-V2] Remove Some Incorrect Comments and Properties in ElasticsearchCommitInfo
• [Hotfix][Jdbc] Fix table/query columns order merge for jdbc catalog (#6771)
• [Fix][KuduCatalogFactory]: Fix KuduCatalogFactory.optionRule() will throw an Exception (#6787)
• [Bug Fix] Sink Doris error status(#6753) (#6755)
• [Bug][Improve][Connector-v2][ElasticsearchSource] Fix behavior when source empty，Support SourceConfig.SOURCE field empty. (#6425)
• [Fix] Fix Oracle type converter handle negative scale in number type (#6758)
• [HiveSink]Fix the risk of resource leakage. (#6721)
• [bigfix][S3 File]:Change the [SCHEMA] attribute of the [S3CONF class] to be non-static to avoid being reassigned after deserialization (#6717)
• [Fix][Connector-v2] Fix the sql statement error of create table for doris and starrocks (#6679)
• [Fix][connector-rocketmq] commit a correct offset to broker & reduce ThreadInterruptedException log (#6668)
• [Fix][Connector-V2] Fix doris TRANSFER_ENCODING header error (#7267)

Transformer

• [Fix][Transform] Fix sql transform lose source type and options (#6855)

Zeta(ST-Engine)

• [Hotfix][Zeta] Fix taskgroup failed log lost (#7241)
• [Fix][Zeta] Fix release slot resource twice (#7236)
• [Fix][Zeta] Fix hybrid deployment can not get worker when init (#7235)
• [HotFix] [Remote Debug] Description When remote debug is enabled, the Seatunel Server 5801 port is not listening properly (#7206)
• [Fix][RestApi] Fix hazelcast/rest/maps/submit-job return a wrong jobId (#7154)
• [Hotfix][Zeta] Fix that batch tasks cannot be stopped after retrying (#7083)
• [Fix][Zeta] Fix env jars not working on zeta (#7035)
• [Fix] revert IMAP cache data field type to compatible with old version (#7001)
• [Hotfix][Zeta] Fix generate_client_protocol.sh (#6991)
• [Hotfix][Zeta] Fix job crash when schema changing on multiple parallel readers (#6977)
• [Fix] Fix Metrics lost data bug (#6976)
• [Hotfix][Zeta] Fix okhttp connection leak in event reporter (#6945)
• [Fix] Fix job will never cancel success when JobMaster init not complete (#6885)
• [Hotfix][Zeta] Fix savepoint execute result (#6850)
• [Fix][Zeta] Increase the log level when waitForJobComplete fails (#6827)
• [Fix] Fix MultiTableWriterRunnable can not catch Throwable error (#6734)
• [HotFix][zeta]: Fix loss job (#6759) (#6807)
• [fix][zeta] fix can't release resource issue (#6763)
• [Fix][Script] Fix lose jvm option when jvm_options file not end with \n (#6693)

E2E

• [Fix][E2E] Fix classloader number (#7209)
• [CI] Fix k8s engine it not runs when only test case changed (#7122)
• [Fix] Fix iceberg e2e image tag (#7189)
• [CI][Test] Fix engine-k8s-it failed (#7101)
• [CI] connector-kafka-e2e , connector-rocketmq-e2e module runs repeatedly #7090 (#7091)
• [E2E][HBase]Refactor hbase e2e (#6859)
• [Fix] Fix ConnectorSpecificationCheckTest failed (#6828)
• [CI] Fix FixSlotResourceTest testNotEnoughResource test error (#6820)

Improve

Core

• [Improve][Core] Move MultiTableSink to seatunnel-api module (#7243)
• [Improve] Bump org.xerial.snappy:snappy-java (#7144)
• [Improve] delete log for protect user privacy (#6987)
• [Improve][Log] Auto refresh log configuration (#6915)
• [Improve] [shell] Set the env SEATUNNEL_HOME automatically (#6924)
• [Improve] Optimize log display during startup command (#6816)

Connectors

• [Improve][Doris Connector] Unified serialization method,Use RowToJsonConverter and TextSerializationSchema (#7229)
• [Improve][Jdbc] Support write unicode text into sqlserver (#7159)
• [Improve] Improve error message when can not parse datetime value (#7181)
• [Improve][Hive] Close resources when exception occurs (#7205)
• [Improve][Jdbc] Remove user info in catalog-table options (#7178)
• [Improve][Connector-V2] Add support for MySQL 8.4 (#7151)
• [Improve][hbase] The specified column is written to the specified column family (#5234)
• [Improve] Change catalog table log to debug level (#7136)
• [Improve][Connector-V2] Support schema evolution for mysql-cdc and mysql-jdbc (#6929)
• [Improve][Redis] Redis reader use scan cammnd instead of keys, single mode reader/writer support batch (#7087)
• [Improve][Connector-V2][MongoDB] Support to convert to double from any numeric type (#6997)
• [Improve][[Jdbc]sink sql support custom field.(#6515) (#6525)
• [Improve][Connector-V2][Iceberg] Add hadoop s3 catalog e2e testcase (#5745)
• [Improve][Connector-V2] Clean key name in catalog table (#6942)
• [Improve][File Connector]Improve xml read code & fix can not use true for a boolean option (#6930)
• [Improve][Files] Support write fixed/timestamp as int96 of parquet (#6971)
• [Improve][Paimon] Add check for the base type between source and sink before write. (#6953)
• [Improve][Connector-V2] Improve the paimon source (#6887)
• [Improve] Make ClickhouseFileSinker support tables containing materialized columns (#6956)
• [Improve][Jdbc] Quotes the identifier for table path (#6951)
• [Improve][CDC] Bump the version of debezium to 1.9.8.Final (#6740)
• [Improve][Connector-V2] Support hive catalog for paimon sink (#6833)
• [Improve][CDC] Close idle subtasks gorup(reader/writer) in increment phase (#6526)
• [Improve][JDBC Source] Fix Split can not be cancel (#6825)
• [Improve] Add conditional of start.mode with timestamp in mongo cdc option rule (#6770)
• [Improve] Improve doris create table template default value (#6720)
• [Improve] [Clickhouse] Remove check when set allow_experimental_lightweight_delete false(#6727) (#6728)
• [Improve] Improve read table schema in cdc connector (#6702)
• [Improve][mysql-cdc] Fallback to desc table when show create table failed (#6701)
• [Improve][Connector-v2] The hive connector support multiple filesystem (#6648)
• [Improve][Hive] udpate hive3 version (#6699)
• [Improve][mysql-cdc] Support mysql 5.5 versions (#6710)
• [Improve] Update aws-java-sdk-budle version to fix job can not run bug
• [Improve] Improve doris stream load client side error message (#6688)
• [Improve][Connector] Add some sqlserver IDENTITY type for catalog (#6822)
• [Improve][Connector-v2] Optimize the count table rows for jdbc-oracle and oracle-cdc (#7248)

Transformer

• [Improve][Transform-V2] Remove SQL query validation on table name (#7020)

Zeta(ST-Engine)

• [Improve][Restapi] Add ip and port to monitoring information (#7203)
• [Improve][Zeta] Add check for submit duplicate job id (#7021)
• [Improve][Zeta] Add retry when request slot on resource manager (#7049)
• [Improve][Zeta] Fix JobMaster reset app classloader twice (#7063)
• [Improve][Engine] Support custom job id for rest-api named submit-job (#7053)
• [Improve][Zeta] Clean checkpoint file when job FINISHED/CANCELED (#6938)
• [Improve][Zeta] Support close idle task for multiple sinks (#6864)
• [Improve][Zeta] Move SaveMode behavior to master (#6843)
• [Improve] Add custom job id arg in client (#6943)
• [Improve][Zeta] when job finished...

2.3.5 Release

2.3.5 Release Note

Bug fix

Core

• [fix] fix get seatunnel row size npe issue (#6681)
• [Hotfix] Fix DEFAULT TABLE problem (#6352)
• [Chore] Fix file spell errors (#6606)
• [BugFix][Spark-translation] map type cast error (#6552)
• [Hotfix] Fix spark example (#6486)
• [Hotfix] Fix compile error (#6463)

Transformer

• [Fix][SQLTransform] fix the scale loss for the sql transform (#6553)
• [Bug] Fix minus constant error in SQLTransform (#6533)

Connectors

• [Fix][Kafka-Sink] fix kafka sink factory option rule (#6657)
• [Hotfix] fix http source can not read yyyy-MM-dd HH:mm:ss format bug & Improve DateTime Utils (#6601)
• [Bug] Fix OrcWriteStrategy/ParquetWriteStrategy doesn't login with kerberos (#6472)
• [Fix][Doc] Fix FTP sink config key username to user (#6627)
• [E2E] Fix AmazondynamodbIT unstable (#6640)
• [Fix][Connector-V2] Fix add hive partition error when partition already existed (#6577)
• [Fix][Connector-V2] Fixed doris/starrocks create table sql parse error (#6580)
• [Fix][Connector-V2] Fix doris sink can not be closed when stream load not read any data (#6570)
• [Fix][Connector-V2] Fix connector support SPI but without no args constructor (#6551)
• [Fix][Connector-V2] Fix doris source select fields loss primary key information (#6339)
• [Fix][FakeSource] fix random from template not include the latest value issue (#6438)
• [Fix][Connector-V2] Fix mongodb cdc start up mode option values not right (#6338)
• [BugFix][Connector-file-sftp] Fix SFTPInputStream.close does not correctly trigger the closing of the file stream (#6323) (#6329)
• [Fix] Fix doris stream load failed not reported error (#6315)
• [fix][connector-rocketmq]Fix a NPE problem when checkpoint.interval is set too small(#6624)
• [Bugfix][TDengine] Fix the issue of losing the driver due to multiple calls to the submit job REST API #6581 (#6596)
• [Fix][StarRocks] Fix NPE when upstream catalogtable table path only have table name part (#6540)

Formats

• [Bug] [formats] Fix fail to parse line when content contains the file delimiter (#6589)

Zeta(ST-Engine)

• [Hotfix] fix http source can not read yyyy-MM-dd HH:mm:ss format bug & Improve DateTime Utils (#6601)
• [Fix][Zeta] Fix the thread stuck problem caused by savepoint checking mechanism (#6568)
• [Fix][Zeta] improve the local mode hazelcast connection (#6521)
• [Fix][Zeta] Fix thread classloader be set to null when use cache mode (#6509)
• [Bug] [zeta] Fix null pointer exception when submitting jobs (#6492)
• [bugfix] [Zeta] Fix the problem of class loader not releasing when using REST API to submit jobs
• [BUG][Zeta]job name Display error #6470
• [Hotfix][Zeta] Fix job deadlock when schema change (#6389)

E2E

• [E2E] Enable StarRocksCDCSinkIT (#6626)

Improve

• [Doc][Improve] Add Support Chinese for seatunnel-engine (#6656)
• [Doc][Improve]Add Support Chinese for start-v2/locally/quick-start-flink.md and start-v2/locally/quick-start-spark.md (#6412)
• [Improve] add icon for idea (#6394)
• [Improve] Add deprecated annotation for ReadonlyConfig::toConfig (#6353)

Core

• [Improve][RestAPI] always return jobId when call getJobInfoById API (#6422)
• [Improve][RestAPI] return finished job info when job is finished (#6576)
• [Improve] Improve MultiTableSinkWriter prepare commit performance (#6495)
• [Improve] Add SaveMode log of process detail (#6375)
• [Improve][API] Unify type system api(data & type) (#5872)

Formats

• [Improve] Improve read with parquet type convert error (#6683)

Connector-V2

• [Improve][Connector-V2]Support multi-table sink feature for redis (#6314)
• [Improve][Connector-V2] oracle cdc e2e optimization (#6232)
• [Improve][Connector-V2]Support multi-table sink feature for httpsink (#6316)
• [Improve][Connector-V2] Support INFINI Easysearch (#5933)
• [Improve][Connector-V2] Support hadoop ha and kerberos for paimon sink (#6585)
• [Improve][CDC-Connector]Fix CDC option rule. (#6454)
• [Improve][CDC] Optimize memory allocation for snapshot split reading (#6281)
• [Improve][Connector-V2] Support TableSourceFactory on StarRocks (#6498)
• [Improve][Jdbc] Using varchar2 datatype store string in oracle (#6392)
• [Improve] StarRocksSourceReader use the existing client (#6480)
• [Improve][JDBC] Optimized code style for getting jdbc field types (#6583)
• [Improve][Connector-V2] Add ElasticSearch type converter (#6546)
• [Improve][Connector-V2] Support read orc with schema config to cast type (#6531)
• [Improve][Jdbc] Support custom case-sensitive config for dameng (#6510)
• [Improve][Jdbc] Increase tyepe converter when auto creating tables (#6617)
• [Improve][CDC] Optimize split state memory allocation in increment phase (#6554)
• [Improve][CDC] Improve read performance when record not contains schema field (#6571)
• [Improve][Jdbc] Add quote identifier for sql (#6669)
• [Improve] Add disable 2pc in SelectDB cloud sink (#6266)
• [Doc][Improve] Add Support Kerberos Auth For Kafka Connector #6653

CI

• [CI] Fix error repository name in ci config files (#4795)

Zeta(ST-Engine)

• [Improve][Zeta] Add classloader cache mode to fix metaspace leak (#6355)
• [Improve][Test] Fix test unstable on ResourceManger and EventReport module (#6620)
• [Improve][Test] Run all test when code merged into dev branch (#6609)
• [Improve][Test] Make classloader cache testing more stable (#6597)
• [Improve][Zeta][storage] update hdfs configuration, support more parameters (#6547)
• [Improve][Zeta]Optimize the logic of RestHttpGetCommandProcessor#getSeaTunnelServer() (#6666)

Transformer

• [Improve][Transform] Sql transform support inner strucy query (#6484)
• [Improve][Transform] Remove Fallback during parsing Transform process (#6644)
• [Improve][Transform] Remove can't find field exception (#6691)

Feature

Core

• [Feature][Tool] Add connector check script for issue 6199 (#6635)
• [Feature][Core] Support listening for message delayed events in cdc source (#6634)
• [Feature][Core] Support event listener for job (#6419)

Connector-V2

• [Feature][connector-v2] add xugudb connector (#6561)
• [Feature][Connector-V2] Support multi-table sink feature for paimon #5652 (#6449)
• [Feature][Connectors-V2][File]support assign encoding for file source/sink (#6489)
• [Feature][Connector]update pgsql-cdc publication for add table (#6309)
• [Feature][Paimon] Support specify paimon table write properties, partition keys and primary keys (#6535)
• [Feature][Feature] Support nanosecond in Doris DateTimeV2 type (#6358)
• [Feature][Feature] Support nanosecond in SelectDB DateTimeV2 type (#6332)
• [Feature][Feature] Supports iceberg sink #6198 (#6265)

Zeta(ST-Engine)

• [Zeta] Support config job retry times in job config (#6690)

Docs

• [Docs] fix kafka format typo (#6633)
• [Fix][Doc] Fixed links in some documents (#6673)
• [Fix][Doc] Fix some spell errors (#6628)
• [Fix][Doc] Fixed typography error in starrocks sink document (#6579)
• [Hotfix][Doc][Chinese] Fix invalid link about configure logging related parameters (#6442)
• [Fix][Doc]Seatunnel Engine/checkpoint-storage.md doc error(#6369)

2.3.4 Release

2.3.4 Release Note

Bug Fix

Core

• [Core] [API] Fixed generic class loss for lists (#4421)
• [Starter] Fix the problem of "," being divided in [] (#5401)
• [Core] [API] Fix ReadonlyConfig lose key error (#5565)
• [Core] [API] Fix get bytes from LinkHashMap (#5622)
• [Core] [API] Fix log error when multi-table sink close (#5683)
• [Core] [API] Fix MultiTableSink return committer but sink do not support (#5710)
• [Core] [API] Fix the error msg when parse schema with unsupported type (#5790)
• [Core] [API] Fix flaky test OptionUtilTest.test (#5894)
• [Core] [API] Fix SaveModeHandler not be closed (#5843)
• [Core] [API] Fix MultiTableSinkWriter thread index always 1 (#5832)
• [Core] [API] Fix SeaTunnelRow::getBytesSize not support map interface (#5990)
• [Core] [Common] Fix FileUtils::createNewFile not create new file (#5943)
• [Core] [API] Fix Debezium format cannot parse date/time/timestamp (#5887)
• [Starter] When inside double quotes,',' are treated as normal characters instead of delimiters (#6042
• [Core] [Common] Replace CommonErrorCodeDeprecated.JSON_OPERATION_FAILED (#5978)
• [Core] [API] Fix Object.class option value can not return normal value (#6247)

Transformer-V2

• [All] Fix PrimaryKey in transform (#5704)
• [All] Fix cast to timestamp, date, time bug (#5812)

Formats

• [Text] Allow the entry in the map to be null and allow the key in the entry to be null (#5277)

Connector-V2

• [Connector-V2] [Clickhouse] Fix clickhouse old version compatibility (#5326)
• [Connector-V2] [Clickhouse] Fix http header cover (#5446)
• [Connector-V2] [StarRocks] Fix starrocks template sql parser (#5332)
• [Connector-V2] [Hive] Fix the bug that hive-site.xml can not be injected in HiveConf (#5261)
• [Connector-V2] [Clickhouse] Fix clickhouse sink flush bug (#5448)
• [Connector-V2] [Hive] Fix An error occurred reading an empty directory (#5427)
• [Connector-V2] [Hive] Fix An error occurred reading an empty directory (#5427)" (#5487)
• [Connector-V2] [Oss jindo] Fix the problem of jindo driver download failure. (#5511)
• [Connector-V2] [Oss jindo] Remove useless code (#5540)
• [Connector-V2] [File] Fix WriteStrategy parallel writing thread unsafe issue (#5546)
• [Connector-V2] [CDC] Fix the cdc bug about NPE when the original table deletes a field (#5579)
• [Connector-V2] [Jdbc] oracle catalog create table repeat and oracle pg null point (#5517)
• [Connector-V2] [CDC] Fix thread-unsafe collection container in cdc enumerator (#5614)
• [Connector-V2] [Mongodb] Fixed unsupported exception caused by bsonNull (#5659)
• [Connector-V2] [File] Fix file sink isPartitionFieldWriteInFile occurred exception when no columns are given (#5508)
• [Connector-V2] [Doris] Fix RestService report NullPointerException (#5319)
• [Connector-V2] [MaxCompute] Fix MaxCompute use not exist SCHEMA option (#5708)
• [Connector-V2] [Doris] Using try-with-resources to simplify the code. (#4995)
• [Connector-V2] [Clickhouse] Fixed an out-of-order BUG with output data fields of clickhouse-sink (#5346)
• [Connector-V2] [Jdbc] support postgresql xml type (#5724)
• [Connector-V2] [Jdbc] Nullable Column source have null data could be unexpected results. (#5560)
• [Connector-V2] [Iceberg] Iceberg source lost data with parallelism option (#5732)
• [Connector-V2] [Jdbc] Fix PG will not create index when using auto create table #5721
• [Connector-V2] [Jdbc] Fix database identifier (#5756)
• [Connector-V2] [CDC] Fix MultiTableSink restore failed when add new table (#5746)
• [Connector-V2] [Postgres CDC] Fix Postgres create table test case failed (#5778)
• [Connector-V2] [CDC] Clean unused code (#5785)
• [Connector-V2] [CDC] Fix state recovery error when switching a single table to multiple tables (#5784)
• [Connector-V2] [ElasticSearch] Fixed conversion exception of elasticsearch array format (#5825)
• [Connector-V2] [Jdbc] Fix read from Oracle Date type value lose time (#5814)
• [Connector-V2] [Pulsar] Fix: update IDENTIFIER = Pulsar for pulsar-datasource on project:seatunnel-web (#5852)
• [Connector-V2] [Jdbc] Fix Hive-Jdbc use krb5 overwrite kerberosKeytabPath (#5891)
• [Connector-V2] [InfluxDB] Resolve invalid SQL in initColumnsIndex method caused by direct QUERY_LIMIT appendage with 'tz' function. (#4829)
• [Connector-V2] [Jdbc] Fix cdc updates were not filtering same primary key (#5923)
• [Connector-V2] [File] Parquet reader parsing array type exception. (#4457)
• [Connector-V2] [Http] Fix bug http config no schema option and improve e2e test add case (#5939)
• [Connector-V2] [Doris] Fix DorisCatalog not implement name method (#5988)
• [Connector-V2] [TDengine] Fix the degree of multiple parallelism affects driver loading (#6020)
• [Connector-V2] [Jdbc] Fix jdbc setFetchSize error (#6005)
• [Connector-V2] [CDC] Fix CDC job cannot consume incremental data After restore run (#625) (#6094)
• [Connector-V2] [File] Fix the Issue of Abnormal Data Reading from Excel Files (#5932)
• [Connector-V2] [CDC] Fix NPE caused by adding a table for restore job (#6145)
• [Connector-V2] [Jdbc] Fix dameng catalog query table sql (#6141)
• [Connector-V2] [Jdbc] update pgsql catalog for save mode (#6080)
• [Connector-V2] [Jdbc] Fix Spliter Error in Case of Extensive Duplicate Data (#6026)
• [Connector-V2] [CDC] Fix added columns cannot be parsed after job restore (#6118)
• [Connector-V2] [CDC] Fix negative values in CDCRecordEmitDelay metric (#6259)
• [Connector-V2] [Oracle CDC] Fix invalid split key when no primary key (#6251)

Zeta(ST-Engine)

• [Zeta] Fix NotifyTaskRestoreOperation npe (#5362)
• [Zeta] Fix Zeta will close task twice error (#5422)
• [Zeta] Disable CheckpointTimeOutTest (#5438)
• [Zeta] Fix CDC task restore throw NPE (#5507)
• [Zeta] Multiple sink actions of the same type have the same name (#5499)
• [Zeta] Checkpoint exception status messages exclude state data (#5547)
• [Zeta] Fix memory leak issues related to checkpoints (#5539)
• [Zeta] Fix the checkpoint be blocked with long time (#5695)
• [Zeta] Fix the problem of unstable job status (#5450)
• [Zeta] Fix submit job api (#5702)
• [Zeta] Set default DeployMode to DeployMode.CLIENT (#5783)
• [Zeta] rest api submit a job with chinese name return Garbled code name (#5870)
• [Zeta] Fix CheckpointCoordinator report NPE when ack not existed pending checkpoint (#5909)
• [Zeta] Fix submit job has the same job name error. (#6041)
• [Zeta] Fixed the problem that the return list is empty due to no status parameters (#6040)
• [Zeta] Fix zeta scheduler bug (#6050)
• [Zeta] Fix job can not restore when last checkpoint failed (#6193)
• [Zeta] [Rest-API] Submit or stop job from an inactive master node (#6217)

E2E

• [E2E] [Common] Update test container version of seatunnel engine (#5323)
• [E2E] [Jdbc] Fix not remove docker image after test finish on jdbc suite (#5586)
• [E2E] [ClickHouse] Enhance ClickHouse E2E testing to trigger multiple checkpoints (#5476)
• [E2E] Fix not remove docker image after test finish on jdbc suite (#5586)
• [E2E] Fix ConnectorPackageServiceContainer miss implement getSavePointCommand/getRestoreCommand (#5780)
• [E2E] Fix build failed cause by JdbcHiveIT and SparkSinkTest (#5798)
• [E2E] Fix submit job case error (#6059)
• [E2E] Fixed action related err (#6264)
• [E2E] Lock mysql container version to 8.0 (#6263)

CI

• [CI] Fix jindo oss connector name (#5385)
• [Build] Fix error msg when fork repository not up to date. (#5497)
• [CI] Fix file change not be check when CI run in fork repository (#5515)
• [CI] remove jindo dependencies (#5528)
• [CI] Fix phoenix ci error (#5530)
• [Build] Update build version to 2.3.4-SNAPSHOT (#5619)
• [Build] Ensure install-plugin.sh compatibility with sh on Debian #5630 (#5631)
• [CI] [Chore] Remove useless sonar check script (#5665
• [Chore] Remove DISCLAIMER file (#5673)
• [CI] Fix CI unstable problem (#5896)
• [Build] Fix empty line in config/plugin_config causes BUILD FAILURE (#5921)
• [CI] Fix CI not run Kudu/AmazonSQS IT when not change api (#5955)
• [CI] Split doris e2e into separate modules (#5999)
• [CI] Fix Dead Links checker failure (#6016)
• [CI] Fix e2e error (#6018)
• [Build] Updated pom.xml (#6113)
• [Build] Solve the problem of example running failure (#6173)
• [Build] Fix build error (#6196)
• [CI] Fix engine client not close  (#6241)

Examples

• [Examples] modify the transform URL link (#5298)

Improve

• [Improve][CheckStyle] Remove useless 'SuppressWarnings' annotation of checkstyle. (#5260)
• [Improve][CheckStyle] Adjust the phase of spotless plugin for release plugin. (#5607)

Core

• [Core] [API] Remove CatalogTable field in CatalogTableUtil (#5521)
• [Core] [API] Move get schema logic from Config to ReadonlyConfig (#5534)
• [Starter] Throw IllegalArgumentException when find multiple connector jar for one pluginIdentifier (#5551)
• [Core] [API] Refactor CatalogTable and add SeaTunnelSource::getProducedCatalogTables (#5562)
• [Core] [API] Support config column/primaryKey/constraintKey in schema (#5564)
• [Core] [API] Remove useless ReadonlyConfig flatten feature (#5612)
• [Core] [Flink & Spark] Refactor Spark/Flink execution processor (#5595)
• [Core] [API] Mark SeaTunnelPluginLifeCycle as deprecated (#5625
• [Core] [API] Support config tableIdentifier for schema (#5628)
• [Core] [Pom] Add junit4 to the root pom (#5611)
• [Core] [API] Remove catalog tag for config file (#5645)
• [Core] [API] Remove useless transform code come from setTypeInfo (#5647)
• [Core] [API] Make sure CatalogTable options and partitionKeys are mutable (#5681)
• [Core] [API] Add default implement for SeaTunnelSource::getProducedType (#5670)
• [Core] [API] Add default implement for SeaTunnelSink::setTypeInfo (#5682)
• [Core] [API] Add warning for use fall back keys (#5753)
• [Core] [API] Adjust the sleep mode of flink and spark e...

2.3.3 Release

2.3.3 Release Note

Bug fix

Core

• [Translation] [Spark] Fix SeaTunnelRowConvertor fail to convert when schema contains row type (#5170)

Connector

• [Connector] [Jdbc] Fix the problem of JdbcOutputFormat database connection leak (#4802)
• [Connector-V2] [AmazonDynamoDB] Fix the problem that all table data cannot be obtained (#5146)
• [Connector-V2] [CDC-base] Fix cdc connection does not close (#4922)
• [Connector-V2] [Clickhouse] Fix clickhouse source connector read Nullable() type is not null,example:Nullable(Float64) while value is null the result is 0.0 (#5080)
• [Connector-V2] [ClickhouseFile] Fix ClickhouseFile write file failed when field value is null (#4937)
• [Connector-V2] [Doris] Add streamload Content-type for doris URLdecode error (#4880)
• [Connector-V2] [E2E] Fix maven scope (#4901)
• [Connector-V2] [File-Base] Fix Hadoop Kerberos authentication related issues (#5171)
• [Connector-V2] [Flink-runtime] Flink register table Environment The running mode is set tojob.mode (#4826)
• [Connector-V2] [HbaseSink] Fix default timestamp (#4958)
• [Connector-V2] [Hive] Support user-defined hive-site.xml (#4965)
• [Connector-V2] [Jdbc] Support postgresql json type (#5194)
• [Connector-V2] [Jdbc] Fix XA Transaction bug (#5020)
• [Connector-V2] [Jdbc] Modify error comment in JdbcSourceConfigFactory (#5088)
• [Connector-V2] [Jdbc-Oracle] Fix the Oracle number type mapping problem (#5209)
• [Connector-V2] [Jdbc-Postgres] PostgreSQL versions below 9.5 are compatible use cdc sync problem (#5120)
• [Connector-V2] [JindoOssFile] Fix plugin-mapping.properties (#5215)
• [Connector-V2] [LocalFileSink] Fix LocalFile Sink file_format_type (#5118)
• [Connector-V2] [Mongodb] Fix document error content and remove redundant code (#4982)
• [Connector-V2] [Mongodb] Compatible with historical parameters (#4997)
• [Connector-V2] [MongoDB-CDC] Incremental data kind error in snapshot phase (#5184)
• [Connector-V2] [MongoDB-CDC] Solve startup resume token is negative (#5143)
• [Connector-V2] [MongoDB-CDC] Refine data format to adapt to universal logic (#5162)
• [Connector-V2] [MongoDB-CDC] Fix array index anomalies caused by #5057 (#5195)
• [Connector-V2] [MySql-CDC] Fix jdbc connection leak for mysql (#5037)
• [Connector-V2] [MySql-CDC] Fix mysql bit column to java byte (#4817)
• [Connector-V2] [MySql-CDC] Fix listener not released when BinlogClient reuse (#5011)
• [Connector-V2] [MySql-CDC] ServerId from int to long (#5033) (#5035)
• [Connector-V2] [Paimon] Bump paimon-bundle version to 0.4.0-incubating (#5219)
• [Connector-V2] [Redis] Fix redis nodes format error (#4981)
• [Connector-V2] [StarRocks] Fix StarRocksJsonSerializer will transform array/map/row to string (#5281)
• [Connector-V2] [StarRocks] Fix code style (#4966)

Zeta(ST-Engine)

• [Zeta] Resolved the issue causing checkpoints to halt on tolerable-failure=0 (#5263)
• [Zeta] Fix the issue of two identical IDs appearing when executing seatunnel.sh -l as the job resumes (#5191)
• [Zeta] Fix SinkFlowLifeCycle without init lastCommitInfo (#5152)
• [Zeta] Fix MultipleTableJobConfigParser ignore env option (#5067)
• [Zeta] Fix checkpoint error report without msg (#5137)
• [Zeta] [REST-API] Fix class not found exception (#5063)
• [Zeta] [REST-API] Fix browser long type intercept (#5267)
• [Zeta] Fix task state memory leak (#5139)
• [Zeta] Fix when execute seatunnel.sh, console may print error (#4984)
• [Zeta] Fixed the issue of duplicated metrics caused by job fault tolerance or restore (#5214)
• [Zeta] Avoid Redundant Job Submissions by Checking Job Status (#5229)
• [Zeta] Fix the serialization issue of GetMetricsOperation during multi-node operation (#5206)
• [Zeta] Fix after the savepoint job is restored, the checkpoint file cannot be generated #4985 (#5051)
• [Zeta] Fixed multi-table job data loss and latency issues (#149) (#5031)
• [Zeta] Fix the HdfsStorage can not delete checkpoint file #5046 (#5054)
• [Zeta] Fix if the initialization task throws an error and the task can not return to master (#4888)

TEST

• [Test] Split updated modules integration test for part 5 (#5208)
• [Test] Add code style check when docs changed (#5183)
• [Test] Turn on the testSavepoint test (#5199)
• [Test] Fix the CI Job name error (#5032)
• [Test] [E2E] Fix testGetErrorInfo case error (#5282)
• [Test] [Kafka] Kafka e2e error (#5200)
• [Test] [MongoDB-CDC] Fix mongodb cdc e2e instability (#5128)
• [Test] Declare files that will always have UNIX line endings on checkout (#5221)
• [Test] Fixed a minor bug (#5274)
• [Test] Fixed IMap file storage e2e bug (#5237)
• [Test] Split updated modules integration test for part 4 (#5028)
• [Test] Fix zeta ci error (#5254)

Others

• Update the project description (#4967)
• Update version to 2.3.3-SNAPSHOT (#5043)
• Fix com.google.common.base.Preconditions to seatunnel shade one (#5284)
• Fix style error (#5280)
• Fix code style (#5092)

Improve

Core

• [API & Zeta] Using connector custom serializer encode/decode states (#5238)
• [API]Improve savemode api (#4767)

Connector

• [CDC Base] Add some split parameters to the optionRule (#5161)
• [CDC Base] Support exactly-once of cdc and fix the BinlogOffset comparing bug (#5057)
• [CDC Base] Add a fast sampling method that supports character types (#5179)
• [CDC Base] Change driver scope to provider (#5002)
• [CDC Base] Remove driver for cdc connector (#4952)
• [Connector-V2] [File] Optimize files commit order (#5045)
• [Connector-V2] [HiveSink] Remove drop partition when abort (#4940)
• [Connector-V2] [Jdbc] Remove scheduler in JDBC sink #4736 (#5168)
• [Connector-V2] [Jdbc] Check url not null throw friendly message (#5097)
• [Connector-V2] [Jdbc] Support for Decimal types as splict keys (#4634)
• [Connector-V2] [Mongodb] Sink support transaction update/writing (#5034)
• [Connector-V2] [Mongodb] Optimize reading logic (#5001)
• [Connector-V2] [Neo4j] Supports neo4j sink batch write and update docs (#4841)
• [Connector-V2] [OSS-Jindo] Optimize jindo oss connector (#4964)
• [Connector-V2] [Tablestore sink] Remove scheduler in Tablestore sink (#5272)

Transform

• [Transform-V2] [SQL] Support use catalogTableName as SQL expression (#5273)

CI

• [CI/CD] Remove 'paths-ignore', enable the code style check for markdown files (#5286)

Zeta(ST-Engine)

• [Zeta] Upgrade the key log output level (#4993)
• [Zeta] Don't trigger handleSaveMode when restore (#5192)
• [Zeta] Move checkpoint notify complete in checkpoint stage (#5185)
• [Zeta] Add sleep for Task to reduce CPU cost (#5117)
• [Zeta] Add an interface for batch retrieval of JobMetrics (#4576)
• [Zeta] Move imap storage file dependency packages to submodules (#5218)
• [Zeta] Improve CheckpointCoordinator log error when report error from task (#178) (#5134)
• [Zeta] Improve CheckpointCoordinator notify complete when restore (#5136)

Feature

Connector

• [Connector-V2] [CDC-base] Support tables without primary keys (with unique keys) (#163) (#5150)
• [Connector-V2] [CDC-base] Support disable/enable exactly once for INITIAL (#4921)
• [Connector-V2] [CDC-base] Change the time zone to the default time zone (#5030)
• [Connector-V2] [CDC-base] Support string type shard fields (#5147)
• [Connector-V2] [Clickhouse] Add clickhouse connector time zone key,default system time zone (#5078)
• [Connector-V2] [Clickhouse] Clickhouse writes with checkpoints (#4999)
• [Connector-V2] [Doris] Add doris factory type convertor (#5061)
• [Connector-V2] [File] Add config of 'file_filter_pattern', which used for filtering files (#5153)
• [Connector-V2] [File] Add cos source&sink (#4979)
• [Connector-V2] [Hive] Hive add option because web need (#5154)
• [Connector-V2] [Jdbc] Add JDBC Catalog auto create table (#4917)
• [Connector-V2] [Jdbc] Add oceanbase dialect factory (#4989)
• [Connector-V2] [Jdbc] JDBC source support string type as partition key (#4947)
• [Connector-V2] [Jdbc-DM] Add DM upsert support (#5073)
• [Connector-V2] [Kafka] Support read debezium format message from kafka (#5066)
• [Connector-V2] [Kafka] Connector-kafka source support data conversion extracted by kafka connect source (#4516)
• [Connector-V2] [Mongodb] Mongodb support cdc sink (#4833)
• [Connector-V2] [Mongodb-CDC] Support source mongodb cdc (#4923)
• [Connector-V2] [Mysql-CDC] Conversion of tinyint(1) to bool is supported (#5105)
• [Connector-V2] [Redis] Support redis to set expiration time (#4975)

Zeta(ST-Engine)

• [Zeta] [CDC-base] Support schema evolution framework(DDL) (#5125)
• [Zeta] [REST-API] Add REST API To Submit Job (#5107)
• [Zeta] Checkpoint support hdfs ha mode (#4942)
• [Zeta] The expiration time of a historical Job can be config (#5180)
• [Zeta] Support history service record job execute error (#5114)
• [Zeta] Configuration files support user variable replacement (#4969)

Transform

• [Transform-V2] [SQL] Support 'select *' and 'like' clause for SQL Transform plugin (#4991)

CI

• [CI] Update action config to support run CI on fork repo (#5065)
• [E2E] [FtpFile] Add ftp file e2e test case (#4647)

Docs

• [Connector-V2][Clickhouse] Reconstruct the clickhouse connector doc (#5085)
• [Connector-V2][Doris] Reconstruct the Doris connector document (#4903)
• [Connector-V2][HDFS] Refactor connector-v2 docs using unified format HDFS (#4871)
• [Connector-V2][Http] Reconstruct the Http connector document (#4962)
• [Connector-V2][Hudi] Reconstruct the Hudi connector document (#4905)
• [Connector-V2][Jdbc] Change the line boundary store value type to BigDecimal (#4900)
• [Connector-V2][Jdbc-DB2、Jdbc-Vertica] Improve DB2 Source Vertica Source & DB2 Sink Vertica Sink document (#5102)
• [Connector-V2][Jdbc-Oracle] Add JDBC Oracle Documentation (#5239)
• [Connector-V2][Kafka] Reconstruct the kafka connector document (#4778)
• [Connector-V2][MyHours] Reconstruct the MyH...

2.3.2 Release

2.3.2 Release Note

Bug fix

Core

• [Core] [API] Fixed generic class loss for lists (#4421)
• [Core] [API] Fix parse nested row data type key changed upper (#4459)

Connector-V2

• [Json-format] [Canal-Json] Fix json deserialize NPE (#4195)
• [Connector-V2] [Jdbc] Field aliases are not supported in the query of jdbc source. (#4210)
• [Connector-V2] [Jdbc] Fix connection failure caused by connection timeout. (#4322)
• [Connector-V2] [Jdbc] Set default value to false of JdbcOption: generate_sink_sql (#4471)
• [Connector-V2] [JDBC] Fix TiDBCatalog without open (#4718)
• [Connector-V2] [Jdbc] Fix XA DataSource crash(Oracle/Dameng/SqlServer) (#4866)
• [Connector-V2] [Pulsar] Fix the bug that can't consume messages all the time. (#4125)
• [Connector-V2] [Eleasticsearch] Document description error (#4390)
• [Connector-V2] [Eleasticsearch] Source deserializer error and inappropriate (#4233)
• [Connector-V2] [Kafka] Fix KafkaProducer resources have never been released. (#4302)
• [Connector-V2] [Kafka] Fix the permission problem caused by client.id. (#4246)
• [Connector-V2] [Kafka] Fix KafkaConsumerThread exit caused by commit offset error. (#4379)
• [Connector-V2] [kafka] Fix the problem that the partition information can not be obtained when kafka is restored (#4764)
• [Connector-V2] [SFTP] Fix incorrect exception handling logic (#4720)
• [Connector-V2] [File] Fix read temp file (#4876)
• [Connector-V2] [CDC Base] Solving the ConcurrentModificationException caused by snapshotState being modified concurrently. (#4877)
• [Connector-V2] [Doris] update last checkpoint id when doing snapshot (#4881)
• [Connector-v2] [kafka] Fix the short interval of pull data settings and revise the format (#4875)
• [Connector-v2] [RabbitMQ] Fix reduplicate ack msg bug and code style (#4842)
• [Connector-V2] [Jdbc] Fix the error of extracting primary key column in sink (#4815)
• [Connector-V2] [Jdbc] Fix reconnect throw close statement exception (#4801)
• [Connector-V2] [Jdbc] Fix sqlserver system table case sensitivity (#4806)
• [Connector-v2] [File] Fix configuration file format and error comments (#4762)
• [Connector-v2] [Jdbc] Fix oracle sql table identifier (#4754)
• [Connector-v2] [Clickhouse] fix get clickhouse local table name with closing bracket from distributed table engineFull (#4710)

Zeta(ST-Engine)

• [Zeta] Fix LogicalDagGeneratorTest testcase (#4401)
• [Zeta] Fix MultipleTableJobConfigParser parse only one transform (#4412)
• [Zeta] Fix missing common plugin jars (#4448)
• [Zeta] Fix handleCheckpointError be called while checkpoint already complete (#4442)
• [Zeta] Fix job error message is not right bug (#4463)
• [Zeta] Fix finding TaskGroup deployment node bug (#4449)
• [Zeta] Fix the bug of conf (#4488)
• [Zeta] Fix Connector load logic from zeta (#4510)
• [Zeta] Fix conflict dependency of hadoop-hdfs (#4509)
• [Zeta] Fix TaskExecutionService synchronized lock will not release (#4886)
• [Zeta] Fix TaskExecutionService will return not active ExecutionContext (#4869)
• [Zeta] Fix deploy operation timeout but task already finished bug (#4867)
• [Zeta] Fix restoreComplete Future can't be completed when cancel task (#4863)
• [Zeta] Fix IMap operation timeout bug (#4859)
• [Zeta] fix pipeline state not right bug (#4823)
• [Zeta] Fix the incorrect setting of transform parallelism (#4814)
• [Zeta] Fix master active bug (#4855)
• [Zeta] Fix completePendingCheckpoint concurrent action (#4854)
• [Zeta] Fix engine runtime error (#4850)
• [Zeta] Fix TaskGroupContext always hold classloader so classloader can't recycle (#4849)
• [Zeta] Fix task notifyTaskStatusToMaster failed when job not running or failed before run (#4847)
• [Zeta] Fix cpu load problem (#4828)
• [zeta] Fix the deadlock issue with JDBC driver loading (#4878)

E2E

• [E2E] [Kafka] Fix kafka e2e testcase (#4520)
• [Container Version] Fix risk of unreproducible test cases #4591
• [E2e] [Mysql-cdc] Removing the excess MySqlIncrementalSourceIT e2e reduces the CI time (#4738)

Improve

Core

• [Core] [Spark] Push transform operation from Spark Driver to Executors (#4503)
• [Core] [Starter] Optimize code structure & remove redundant code (#4525)
• [Core] [Translation] [Flink] Optimize code structure & remove redundant code (#4527)
• [Core] [Starter] Add check of sink and source config to avoid null pointer exception. (#4734)

Connector-V2

• [Connector-V2] [CDC] Improve startup.mode/stop.mode options (#4360)
• [Connector-V2] [CDC] Optimize jdbc fetch-size options (#4352)
• [Connector-V2] [CDC] Fix chunk start/end parameter type error (#4777)
• [Connector-V2] [SQLServer] Fix sqlserver catalog (#4441)
• [Connector-V2] [StarRocks] Improve StarRocks Serialize Error Message (#4458)
• [Connector-V2] [Jdbc] add the log for sql and update some style (#4475)
• [Connector-V2] [Jdbc] Fix the table name is not automatically obtained when multiple tables (#4514)
• [Connector-V2] [S3 & Kafka] Delete unavailable S3 & Kafka Catalogs (#4477)
• [Connector-V2] [Pulsar] Support Canal Format
• [Connector-V2] [CDC base] Implement Sample-based Sharding Strategy with Configurable Sampling Rate (#4856)
• [Connector-V2] [SelectDB] Add a jobId to the selectDB label to distinguish between tasks (#4864)
• [Connector-V2] [Doris] Add a jobId to the doris label to distinguish between tasks (#4839) (#4853)
• [Connector-v2] [Mongodb]Refactor mongodb connector (#4620)
• [Connector-v2] [Jdbc] Populate primary key when jdbc sink is created using CatalogTable (#4755)
• [Transform-V2] Optimize SQL Transform package and Fix Spark type conversion bug of transform (#4490)

CI

• [CI] Fix error repository name in ci config files (#4795)

Zeta(ST-Engine)

• [Zeta] Support run the server through daemon mode (#4161)
• [Zeta] Change ClassLoader To Improve the SDK compatibility of the client (#4447)
• [Zeta] Client Support Async Submit Job (#4456)
• [Zeta] Add more detailed log output. (#4446)
• [Zeta] Improve seatunnel-cluster.sh (#4435)
• [Zeta] Reduce CPU Cost When Task Not Ready (#4479)
• [Zeta] Add parser log (#4485)
• [Zeta] Remove redundant code (#4489)
• [Zeta] Remove redundancy code in validateSQL (#4506)
• [Zeta] Improve JobMetrics fetch performance (#4467)
• [Zeta] Reduce the operation count of imap_running_job_metrics (#4861)
• [Zeta] Speed up listAllJob function (#4852)
• [Zeta] async execute checkpoint trigger and other block method (#4846)
• [Zeta] Reduce the number of IMAPs used by checkpointIdCounter (#4832)
• [Zeta] Cancel pipeline add retry to avoid cancel failed. (#4792)
• [Zeta] Improve Zeta operation max count and ignore NPE (#4787)
• [Zeta] Remove serialize(deserialize) cost when use shuffle action (#4722)

Feature

Core

• [Core] [API] Support convert strings as List option (#4362)
• [Core] [API] Add copy method to Catalog codes (#4414)
• [Core] [API] Add options check before create source and sink and transform in FactoryUtil (#4424)
• [Core] [Shade] Add guava shade module (#4358)

Connector-V2

• [Connector-V2] [CDC] [SQLServer] Support multi-table read (#4377)
• [Connector-V2] [Kafka] Kafka source supports data deserialization failure skipping (#4364)
• [Connector-V2] [Jdbc] [TiDB] Add TiDB catalog (#4438)
• [Connector-V2] [File] Add file excel sink and source (#4164)
• [Connector-v2] [Snowflake] Add Snowflake Source&Sink connector (#4470)
• [Connector-V2] [Pular] support read format for pulsar (#4111)
• [Connector-V2] [Paimon] Introduce paimon connector (#4178)
• [Connector V2] [Cassandra] Expose configurable options in Cassandra (#3681)
• [Connector V2] [Jdbc] Supports GEOMETRY data type for PostgreSQL (#4673)
• [Transform-V2] Add UDF SPI and an example implement for SQL Transform plugin (#4392)
• [Transform-V2] Support copy field list (#4404)
• [Transform-V2] Add support CatalogTable for FieldMapperTransform (#4423)
• [Transform-V2] Add CatalogTable support for ReplaceTransform (#4411)
• [Transform-V2] Add Catalog support for FilterRowKindTransform (#4420)
• [Transform-V2] Add support CatalogTable for FilterFieldTransform (#4422)
• [Transform-V2] Add catalog support for SQL Transform plugin (#4819)

Zeta(ST-Engine)

• [Zeta] Support for mixing Factory and Plugin SPI (#4359)
• [Zeta] Add get running job info by jobId rest api (#4140)
• [Zeta] Add REST API To Get System Monitoring Information (#4315)
• [Transform V2 & Zeta] Make SplitTransform Support CatalogTable And CatalogTable Evolution (#4396)
• [Zeta] Move driver into lib directory and change operation count (#4845)
• [Zeta] Add Metaspace size default value to config file (#4848)
• [Zeta] Reduce the frequency of fetching data from imap (#4851)
• [Zeta] Add OSS support for Imap storage to cluster-mode type (#4683)
• [Zeta] Improve local mode startup request ports (#4660)

Docs

• [Docs] Optimizes part of the Doris and SelectDB connector documentation (#4365)
• [Docs] Fix docs code style (#4368)
• [Docs] Update jdbc doc and kafka doc (#4380)
• [Docs] Fix max_retries default value is 0. (#4383)
• [Docs] Fix markdown syntax (#4426)
• [Docs] Fix Kafka Doc Error Config Key "kafka." (#4427)
• [Docs] Add Transform to Quick Start v2 (#4436)
• [Docs] Fix Mysql sink format doc (#4800)
• [Docs] Add the generate sink sql parameter for the jdbc sink document (#4797)
• [Docs] Add the generate sink sql parameter And example (#4769)
• [Docs] Redshift add defaultRowFetchSize (#4616)
• [Docs] Refactor connector-v2 docs using unified format Mysql (#4590)

2.3.1-release

2.3.1-release

Bug fix

Core

• [Core] [Shade] [Hadoop] Fix hadoop shade dependency can not be used (#3835)
• [Core] Fix Handover using linked blocking queue cause the oom (#3469)
• [Core] Fix a bug of Spark Translation when has a timestamp/date type filed in source (#4226)
• [Core] Remove unnecessary row conversion (#4335)

Connector-V2

• [Connecor-V2] [JDBC] Fix the problem that can not throw exception correctly (#3796)
• [Connector-V2] [JDBC] Fix xa transaction commit failure on pipeline restore (#3809)
• [Connector-V2] [Clickhouse] Fix the serializable problems of Committer (#3803)
• [Connector-V2] [Clickhouse] Fix the performance bug (#3910)
• [Connector-v2] [Clickhouse] Fix clickhouse write cdc changelog update event (#3951)
• [Connector-V2] [Clickhouse] Clickhouse File Connector failed to sink to table with settings like storage_policy (#4172)
• [Connector-V2] [Clickhouse] Clickhouse File Connector not support split mode for write data to all shards of distributed table (#4035)
• [Connector-V2] [File] Fix the error type of option rule path (#3804)
• [Connector-V2] [File] Text And Json WriteStrategy lost the sinkColumnsIndexInRow (#3863)
• [Connector-V2] [Kafka] Fix the bug that can not parse offset format (#3810)
• [Connecor-V2] [Kafka] Fix the default value of commit_on_checkpoint (#3831)
• [Connector-V2] [Kafka] Json deserialize exception log no content (#3874)
• [Connector-V2] [Kafka] Fix commit kafka offset bug. (#3933)
• [Connector-V2] [Kafka] Fix the bug that kafka consumer is not close (#3836)
• [Connector-V2] [Kafka] Fix config option error (#4244)
• [Connector-V2] [CDC] Guaranteed to be exactly-once in the process of switching from SnapshotTask to IncrementalTask (#3837)
• [Connector-V2] [CDC] Fix concurrent modify of splits (#3937)
• [Connector-V2] [CDC] Fix jdbc sink generate update sql (#3940)
• [Connector-V2] [CDC] Fix cdc option rule error (#4018)
• [Connector-V2] [CDC] Fix cdc base shutdown thread not cleared (#4327)
• [Connector-V2] [ALL] Fix ConcurrentModificationException when snapshotState based on SourceReaderBase (#4011)
• [Connector-V2] Fix connector source snapshot state NPE (#4027)
• [Connector-v2] [Pulsar] Fix pulsar option topic-pattern bug (#3989)
• [Connector-V2] [ElasticSearch] Fix es source no data (#4076)
• [Connector-V2] [Hive] Fix hive unknownhost (#4141)
• [Connector-V2] [Pulsar] Fix Pulsar source consumer ack exception (#4237)
• [Connector-V2] [Maxcompute] Fix failed to parse some maxcompute type (#3894)
• [Connector-V2] [Doris] Fix Content Length header already present (#4277)
• [Connector-V2] [ElasticSearch] Fix es field type not support binary (#4274)
• [Connector-V2] [JDBC] Field aliases are not supported in the query of jdbc source (#4210)
• [Json-format] [Canal] Fix json deserialize NPE (#4195)

Zeta(ST-Engine)

• [Zeta] Fix the bug that checkpoint will be continued trigger after job CANCELED (#3808)
• [Zeta] Fix the bug that the source split can not be deserialized in local mode (#3817)
• [Zeta] Fix Engine Metrics will lose when Job be canceled. (#3797)
• [Zeta] Fix CheckpointIDCounter thread not safe problem (#3875)
• [Zeta] Fix SeatunnelChildFirstClassLoader load jackson error (#3884)
• [Zeta] Fix local mode can not generate logs (#3917)
• [Zeta] Fix actionSubtaskState can be null error (#3902)
• [Zeta] A checkpoint exception may be thrown when the active cancel task terminates with an exception (#3915)
• [Zeta] Fix NPE when scheduling sub plan fails. (#3909)
• [Zeta] Fix NPE when starting from checkpoint. (#3904)
• [Zeta] Fix metrics lose on pipeline restart (#3977)
• [Zeta] Fix Job will lost control when JobMaster init failed (#4045)
• [Zeta] Fix ResourceManager Assign Task Not Random (#4078)
• [Zeta] Fix NullPointerException when registering Reader with the enumerator (#4048)
• [Zeta] Fix clean TaskGroupContext Error when target node is offline (#4086)
• [Zeta] Fix Slot Status Not Sync After Restart When All Node Down (#4047)
• [Zeta] Fix parse job mode bug and improve doc (#4091)
• [Zeta] Fix Client Have Error Can't be Shutdown (#4099)
• [Zeta] Fix zeta bugs
• [Zeta] Fix checkpoint storage namespace (#4260)
• [Zeta] Fix read checkpoint file data is incomplete (#4263)
• [Zeta] The pipeline needs to wait for the CheckpointCoordinator to end (#4272)
• [Zeta] Fix CheckpointCoordinator Can't Trigger Timeout Task (#4276)
• [Zeta] Fix job crash when run it on Seatunnel Engine cluster (#4299)
• [Zeta] Fix Default Cluster Not Working In Config File (#3770)
• [Zeta] Adapt StarRocks With Multi-Table And Single-Table Mode (#4324)
• [Zeta] Fix TaskExecutionService Deploy Failed The Job Can't Stop (#4265)
• [Zeta] Fix cancelJob and checkpoint complete error (#4330)

E2E

• [Connector-V2] [SQLServer-CDC] Fix the bug of SQLServer-CDC it case (#3807)
• [Connector-V2] [Clickhouse] Fix the bug of clickhouse e2e case (#3985)

Improve

Core

• [Core] [API] Add parallelism and column projection interface (#3829)
• [Core] [Connector-V2] Add get source method to all source connector (#3846)
• [Core] [Shade] [Hadoop] Improve hadoop shade by including classes in package com.google.common.cache.* (#3858)
• [Core] Use ReadonlyConfig to avoid option being changed (#4056)
• [Core] Give the maven module a human readable name (#4114)
• [Core] Unified the checkpoint setting key of Flink (#4296)

Connector-V2

• [Connector-V2] [MySQL-CDC] Add mysql-cdc source factory (#3791)
• [Connector-V2] [MySQL-CDC] Ennable binlog watermark compare (#4293)
• [Connector-V2] [Kafka] Support user-defined client id (#3783)
• [Connector-V2] [Kafka] Support extract topic from SeaTunnelRow field (#3742)
• [Connector-V2] [Kafka] Add Kafka catalog (#4106)
• [Connector-V2] [JDBC] Remove unused options that in jdbc source factory (#3794)
• [Connector-V2] [JDBC] Support exactly-once semantics for JDBC source connector (#3750)
• [Connector-V2] [JDBC] Improve option rule (#3802) (#3864)
• [Connector-V2] [MongoDB] Support use source query in connector (#3697)
• [Connector-V2] [File] Add the exception stack detail for log output (#3805)
• [Connector-V2] [File] Improve file connector option rule and document (#3812)
• [Connector-V2] [File] Support skip number when reading text csv files (#3900)
• [Connector-V2] [File] Allow the user to set the row delimiter as an empty string (#3854)
• [Connector-V2] [File] Support compress (#3899)
• [Connector-V2] [File & Hive] Support kerberos in hive and hdfs file connector (#3840)
• [Connector-V2] [Hive] Improve config check logic (#3886)
• [Connector-V2] [Hive] Support assign partitions (#3842)
• [Connector-V2] [Hive] Support read text table & Column projection (#4105)
• [Connector-V2] [Clickhouse & Kafka & Rabbitmq & StarRocks] Change connector custom config prefix to map (#3719)
• [Connector-V2] [Clickhouse] Special characters in column names are supported (#3881)
• [Connector-V2] [Clickhouse] Remove Clickhouse Fields Config (#3826)
• [Connector-V2] [Email] Unified exception for email connector (#3898)
• [Connector-V2] [Iceberg] Unified exception for iceberg source connector (#3677)
• [Connector-v2] [StarRocks] Support write cdc changelog event(INSERT/UPDATE/DELETE) (#3865)
• [Connector-V2] [Fake] Improve fake connector (#3932)
• [Connector-V2] [Fake] Optimizing Data Generation Strategies (#4061)
• [Connector-V2] [InfluxDB] Unifie InfluxDB source fields to schema (#3897)
• [Connector-V2] [IoTDB] Unifie IoTDB source fields to schema (#3896)
• [Transform-V2] Add transform factory test (#3887)
• [Connector-V2] [SQLServer-CDC] Add sqlserver cdc optionRule (#4019)
• [Connector-V2] [Elasticsearch] Support https protocol (#3997)
• [Connector-V2] [Elasticsearch] Add ElasticSearch catalog (#4108)
• [Connector-V2] [Elasticsearch] Support dsl filter (#4130)
• [Connector-V2] [S3] Add S3Catalog (#4121)
• [Connector-V2] [Doris] Refactor some Doris Sink code as well as support 2pc and cdc (#4235)
• [Connector-V2] [CDC] Optimize options & add docs for compatible_debezium_json (#4351)

CI

• [CI] Imprve CI/CD process, split all connector it cases to 4 (#3832)
• [CI] Imprve CI/CD process, split all connector it cases to 5 (#4065)
• [E2E] Improve CI stability (#4068)
• [CI] Improve ci steps (#4314) (#4342)

Zeta(ST-Engine)

• [Zeta] Remove unnecessary dependencies in pom (#3795)
• [Zeta] Add the restart job logic when all nodes down (#3784)
• [Zeta] Add the logic that cancel CheckpointCoordinator first before cancel task (#3838)
• [Zeta] Remove seatunnel-api from engine storage (#3834)
• [Zeta] Suppress delete data log level (#4001)
• [Zeta] Set the write data timeout to be configurable (#4059)
• [Zeta] Job clean before JobMaster future complete (#4087)
• [Zeta] Add Slot Sequence To Avoid Active Check Error (#4097)
• [Zeta] Improve Client Job Info Message
• [Zeta] Client Job Info Message Add Order By Submit Time
• [Zeta] JVM parameters distinguish between client and server (#4297)

E2E

• [Connector-V2] [Iceberg] Refactor iceberg connector e2e test cases (#3820)
• [Connector-V2] [Datahub] Refactor DataHub connector e2e test cases (#3866)
• [Connector-V2] [MongoDB] Refactor MongoDB connector e2e test (#3819)
• [Connector-V2] [ES & InfluxDB & Redis] Add a 'nonNull' check to avoid npe when executing 'tearDown'. (#3967)
• [Connector-V2] [Http] Refactor the e2e test of http (#3969)
• [Connector-V2] Remove the use of scala.Tuple in the e2e module (#3974)
• [Zeta] Change E2E To support ClusterFaultToleranceIT (#3976)
• [Zeta] Statistics server job and system resource usage (#3982)
• [Transform-v2] Merge e2e tests and config files of SQL transform plugin (#4278)
• [Connector-V2] [Jdbc] Reactor jdbc e2e with new api, then remove the useless e2e case. For better performance, we split the jdbc e2e module into three modules. (#4165)

Feature

Core

• [Core] [Transform-V2] Support tr...

2.3.0-release

2.3.0-release

Bug fix

Core

• [Core] [Starter] Fix the bug of ST log print failed in some jdk versions #3160
• [Core] [Shell] Fix bug that shell script about downloading plugins does not work #3462

Connector-V2

• [Connector-V2] [Jdbc] Fix the bug that jdbc source can not be stopped in batch mode #3220
• [Connector-V2] [Jdbc] Fix the bug that jdbc connector reset in jdbc connector #3670
• [Connector-V2] [Jdbc] Fix the bug that jdbc connector exactly-once it will throw NullPointerException #3730
• [Connector-V2] [Hive] Fix the following bugs of hive connector: 1. write parquet NullPointerException 2. when restore write from states getting error file path #3258
• [Connector-V2] [File] Fix the bug that when getting file system throw NullPointerException #3506
• [Connector-V2] [File] Fix the bug that when user does not config the fileNameExpression it will throw NullPointerException #3706
• [Connector-V2] [Hudi] Fix the bug that the split owner of Hudi connector may be negative #3184

ST-Engine

• [ST-Engine] Fix bug data file name will duplicate when use SeaTunnel Engine #3717
• [ST-Engine] Fix job restart of all nodes down #3722
• [ST-Engine] Fix the bug that checkpoint stuck in ST-Engine #3213
• [ST-Engine] Fix the bug that checkpoint failed in ST-Engine #3769

E2E

• [E2E] [Spark] Corrected spark version in e2e container #3225

Improve

Core

• [Core] [Starter] [Flink] Upgrade the method of loading extra jars in flink starter #2982
• [Core] [Pom] [Package] Optimize package process #3751

Connector-V1

• [Connector-V1] Remove connector v1 related codes from dev branch #3450

Connector-V2

• [Connector-V2] Add split templates for all connectors #3335
• [Connector-V2] [Redis] Support redis cluster mode & user authentication #3188
• [Connector-V2] [Clickhouse] Support nest type and array type in clickhouse connector #3047
• [Connector-V2] [Clickhouse] Support geo type in clickhouse connector #3141
• [Connector-V2] [Clickhouse] Improve double convert that in clickhouse connector #3441
• [Connector-V2] [Clickhouse] Improve float long convert that in clickhouse connector #3471
• [Connector-V2] [Kafka] Support setting read start offset or message time in kafka connector #3157
• [Connector-V2] [Kafka] Support specify multiple partition keys in kafka connector #3230
• [Connector-V2] [Kafka] Support dynamic discover topic & partition in kafka connector #3125
• [Connector-V2] [Kafka] Support text format for kafka connector #3711
• [Connector-V2] [IotDB] Add the parameter check logic for iotDB sink connector #3412
• [Connector-V2] [Jdbc] Support setting fetch size in jdbc connector #3478
• [Connector-V2] [Jdbc] Support upsert config in jdbc connector #3708
• [Connector-V2] [Jdbc] Optimize the commit process of jdbc connector #3451
• [Connector-V2] [Jdbc] Release jdbc resource when after using #3358
• [Connector-V2] [Oracle] Improve data type mapping of Oracle connector #3486
• [Connector-V2] [Http] Support extract complex json string in http connector #3510
• [Connector-V2] [File] [S3] Support s3a protocol in S3 file connector #3632
• [Connector-V2] [File] [HDFS] Support setting hdfs-site.xml #3778
• [Connector-V2] [File] Support file split in file connectors #3625
• [Connector-V2] [CDC] Support write cdc changelog event in elsticsearch sink connector #3673
• [Connector-V2] [CDC] Support write cdc changelog event in clickhouse sink connector #3653
• [Conncetor-V2] [CDC] Support write cdc changelog event in jdbc connector #3444

ST-Engine

• [ST-Engine] Improve statistic information print format that in ST-Engine #3492
• [ST-Engine] Improve ST-Engine performance #3216
• [ST-Engine] Support user-defined jvm parameters in ST-Engine #3307

CI

• [CI] Improve CI process #3179 #3194

E2E

• [E2E] [Flink] Support execute extra commands on task-manager container #3224
• [E2E] [Jdbc] Increased Jdbc e2e stability #3234

Feature

Core

• [Core] [Log] Integrate slf4j and log4j2 for unified management logs #3025
• [Core] [Connector-V2] [Exception] Unified exception API & Unified connector error tip message #3045
• [Core] [Shade] [Hadoop] Add hadoop shade package for SeaTunnel #3755

Connector-V2

• [Connector-V2] [Elasticsearch] Add elasticsearch source connector #2821
• [Connector-V2] [AmazondynamoDB] Add AmazondynamoDB source & sink connector #3166
• [Connector-V2] [StarRocks] Add StarRocks sink connector #3164
• [Connector-V2] [DB2] Add DB2 source & sink connector #2410
• [Connector-V2] [Transform] Add transform-v2 api #3145
• [Connector-V2] [InfluxDB] Add influxDB sink connector #3174
• [Connector-V2] [Cassandra] Add Cassandra Source & Sink connector #3229
• [Connector-V2] [MyHours] Add MyHours source connector #3228
• [Connector-V2] [Lemlist] Add Lemlist source connector #3346
• [Connector-V2] [CDC] [MySql] Add mysql cdc source connector #3455
• [Connector-V2] [CDC] [SqlServer] Add sqlserver cdc source connector #3686
• [Connector-V2] [Klaviyo] Add Klaviyo source connector #3443
• [Connector-V2] [OneSingal] Add OneSingal source connector #3454
• [Connector-V2] [Slack] Add slack sink connector #3226
• [Connector-V2] [Jira] Add Jira source connector #3473
• [Connector-V2] [Sqlite] Add Sqlite source & sink connector #3089
• [Connector-V2] [OpenMldb] Add openmldb source connector #3313
• [Connector-V2] [Teradata] Add teradata source & sink connector #3362
• [Connector-V2] [Doris] Add doris source & sink connector #3586
• [Connector-V2] [MaxCompute] Add MaxCompute source & sink connector #3640
• [Connector-V2] [Doris] [Streamload] Add doris streamload sink connector #3631
• [Connector-V2] [Redshift] Add redshift source & sink connector #3615
• [Connector-V2] [Notion] Add notion source connector #3470
• [Connector-V2] [File] [Oss-Jindo] Add oss jindo source & sink connector #3456

ST-Engine

• [ST-Engine] Support print job metrics when job finished #3691
• [ST-Engine] Add metrics statistic in ST-Engine #3621
• [ST-Engine] Support IMap file storage in ST-Engine #3418
• [ST-Engine] Support S3 file system for IMap file storage #3675
• [ST-Engine] Support save job restart status information in ST-Engine #3637

E2E

• [E2E] [Http] Add http type connector e2e test cases #3340
• [E2E] [File] [Local] Add local file connector e2e test cases #3221

Docs

• [Docs] [Connector-V2] [Factory] Add TableSourceFactory & TableSinkFactor docs #3343
• [Docs] [Connector-V2] [Schema] Add connector-v2 schema docs #3296
• [Docs] [Connector-V2] [Quick-Manaul] Add error quick reference manual #3437
• [Docs] [README] Improve README and refactored other docs #3619

2.3.0-beta-Release

2.3.0 Beta

[Connector V2]

[New Connector V2 Added]

• [Source] [Kafka] Add Kafka Source Connector (2953)
• [Source] [Pulsar] Add Pulsar Source Connector (1980)
• [Source] [S3File] Add S3 File Source Connector (3119)
• [Source] [JDBC] [Phoenix] Add Phoenix JDBC Source Connector (2499)
• [Source] [JDBC] [SQL Server] Add SQL Server JDBC Source Connector (2646)
• [Source] [JDBC] [Oracle] Add Oracle JDBC Source Connector (2550)
• [Source] [JDBC] [GBase8a] Add GBase8a JDBC Source Connector (3026)
• [Source] [JDBC] [StarRocks] Add StarRocks JDBC Source Connector (3060)
• [Sink] [Kafka] Add Kafka Source Connector (2953)
• [Sink] [S3File] Add S3 File Sink Connector (3119)

[Improve & Bug Fix]

• [Source] [Fake]

  • [Improve] Supports direct definition of data values(row) (2839)
  • [Improve] Improve fake source connector: (2944)
    • Support user-defined map size
    • Support user-defined array size
    • Support user-defined string length
    • Support user-defined bytes length
  • [Improve] Support multiple splits for fake source connector (2974)
  • [Improve] Supports setting the number of splits per parallelism and the reading interval between two splits (3098)

• [Source] [Clickhouse]

  • [Improve] Clickhouse Source random use host when config multi-host (3108)

• [Source] [FtpFile]

  • [BugFix] Fix the bug of incorrect path in windows environment (2980)
  • [Improve] Support extract partition from SeaTunnelRow fields (3085)
  • [Improve] Support parse field from file path (2985)

• [Source] [HDFSFile]

  • [BugFix] Fix the bug of incorrect path in windows environment (2980)
  • [Improve] Support extract partition from SeaTunnelRow fields (3085)
  • [Improve] Support parse field from file path (2985)

• [Source] [LocalFile]

  • [BugFix] Fix the bug of incorrect path in windows environment (2980)
  • [Improve] Support extract partition from SeaTunnelRow fields (3085)
  • [Improve] Support parse field from file path (2985)

• [Source] [OSSFile]

  • [BugFix] Fix the bug of incorrect path in windows environment (2980)
  • [Improve] Support extract partition from SeaTunnelRow fields (3085)
  • [Improve] Support parse field from file path (2985)

• [Source] [IoTDB]

  • [Improve] Improve IoTDB Source Connector (2917)
    • Support extract timestamp、device、measurement from SeaTunnelRow
    • Support TINYINT、SMALLINT
    • Support flush cache to database before prepareCommit

• [Source] [JDBC]

  • [Feature] Support Phoenix JDBC Source (2499)
  • [Feature] Support SQL Server JDBC Source (2646)
  • [Feature] Support Oracle JDBC Source (2550)
  • [Feature] Support StarRocks JDBC Source (3060)
  • [Feature] Support GBase8a JDBC Source (3026)

• [Sink] [Assert]

  • [Improve] 1.Support check the number of rows (2844) (3031):
    • check rows not empty
    • check minimum number of rows
    • check maximum number of rows
  • [Improve] 2.Support direct define of data values(row) (2844) (3031)
  • [Improve] 3.Support setting parallelism as 1 (2844) (3031)

• [Sink] [Clickhouse]

  • [Improve] Clickhouse Support Int128,Int256 Type (3067)

• [Sink] [Console]

  • [Improve] Console sink support print subtask index (3000)

• [Sink] [Enterprise-WeChat]

  • [BugFix] Fix Enterprise-WeChat Sink data serialization (2856)

• [Sink] [FtpFile]

  • [BugFix] Fix the bug of incorrect path in windows environment (2980)
  • [BugFix] Fix filesystem get error (3117)
  • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

• [Sink] [HDFSFile]

  • [BugFix] Fix the bug of incorrect path in windows environment (2980)
  • [BugFix] Fix filesystem get error (3117)
  • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

• [Sink] [LocalFile]

  • [BugFix] Fix the bug of incorrect path in windows environment (2980)
  • [BugFix] Fix filesystem get error (3117)
  • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

• [Sink] [OSSFile]

  • [BugFix] Fix the bug of incorrect path in windows environment (2980)
  • [BugFix] Fix filesystem get error (3117)
  • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

• [Sink] [IoTDB]

  • [Improve] Improve IoTDB Sink Connector (2917)
    • Support align by sql syntax
    • Support sql split ignore case
    • Support restore split offset to at-least-once
    • Support read timestamp from RowRecord
  • [BugFix] Fix IoTDB connector sink NPE (3080)

• [Sink] [JDBC]

  • [BugFix] Fix JDBC split exception (2904)
  • [Feature] Support Phoenix JDBC Source (2499)
  • [Feature] Support SQL Server JDBC Source (2646)
  • [Feature] Support Oracle JDBC Source (2550)
  • [Feature] Support StarRocks JDBC Source (3060)

• [Sink] [Kudu]

  • [Improve] Kudu Sink Connector Support to upsert row (2881)

• [Sink] [Hive]

  • [Improve] Hive Sink supports automatic partition repair (3133)

[Connector V1]

[New Connector V1 Added]

[Improve & Bug Fix]

• [Sink] [Spark-Hbase]
  • [BugFix] Handling null values (3099)

[Starter & Core & API]

[Feature & Improve]

• [Improve] [Sink] Support define parallelism for sink connector (2941)
• [Improve] [all] change Log to @slf4j (3001)
• [Improve] [format] [text] Support read & write SeaTunnelRow type (2969)
• [Improve] [api] [flink] extraction unified method (2862)
• [Feature] [deploy] Add Helm charts (2903)
• [Feature] [seatunnel-text-format] (2884)

[Bug Fix]

• [BugFix] Fix assert connector name error in config/plugin_config file (3127)
• [BugFix] [starter] Fix connector-v2 flink & spark dockerfile (3007)
• [BugFix] [core] Fix spark e...