v2.3.0

Notable changes

• (Snowpipe Streaming) Timestamp of pushing data into Snowflake is added to record_metadata. Note according to the configured kafka buffers, the timestamp value may be have a slight delay between when it is pushed to the Snowpipe Streaming channel buffer and when it is flushed and available on the table.
  • SNOW-1056407 Add SnowflakeConnectorPushTime to record_metadata by @sfc-gh-lshcharbaty in #833
• (Snowpipe Streaming) Optional parameter, snowflake.streaming.closeChannelsInParallel.enabled, for Kafka connector to close channels in parallel which significantly speeds up rebalance time
  • SNOW-1369280 Close channels asynchronously by @sfc-gh-lshcharbaty in #841
• Updated dependencies with known vulnerabilities
  • SGRC-4405 Update Kafka to 3.7.0 and Confluent to 7.6.0 by @sfc-gh-mbobowski in #854

What's Changed

• SNOW-1269024 Installing multiple Java versions for test purposes by @sfc-gh-krosinski in #824
• SNOW-1269025 Testing Confluent kafka connector using multiple JRE versions by @sfc-gh-krosinski in #826
• SNOW-1269038 Stress testing Confluent kafka connector using multiple JRE versions by @sfc-gh-krosinski in #827
• SNOW-1348656 Migrate RecordContentTest to JUnit5 by @sfc-gh-mbobowski in #828
• SNOW-1056407 Cleanup metadata properties class. Migrate records tests to junit5 by @sfc-gh-lshcharbaty in #831
• NO-SNOW Fix build warnings by declaring maven-plugin versions by @sfc-gh-xhuang in #837
• NO-SNOW Fix idempotent multiple column schema evolution by @sfc-gh-xhuang with contribution from @cmonty-paypal in #836
• SNOW-1348656 Send records that could not be parsed to DLQ by @sfc-gh-mbobowski in #830
• SNOW-1056407 Add tests MetaColumnTest for snowpipe streaming by @sfc-gh-lshcharbaty in #832
• SNOW-1369271 Ignore SFException when closing a channel by @sfc-gh-lshcharbaty in #839
• NO-SNOW Remove unused code path from RecordService by @sfc-gh-mbobowski in #838
• NO-SNOW Ugrade to ingest-sdk 2.1.1 by @sfc-gh-xhuang in #843
• SNOW-1167007: Migrate schema when receiving null values for not null cols by @sfc-gh-wtrefon in #849
• SNOW-1061848: Introduce TopicPartitionChannel interface by @sfc-gh-akowalczyk in #851
• SNOW-1061822: Add new properties for streaming client by @sfc-gh-wtrefon in #822
• SNOW-1061848: DirectTopicPartitionChannel - copy of the old implementation with removed buffer by @sfc-gh-akowalczyk in #852
• SNOW-1061848: Fix unit tests by @sfc-gh-akowalczyk in #855
• NO-SNOW Add dependabot by @sfc-gh-mbobowski in #856
• SNOW-1061848: Run TopicPartitionChannelIT against a non-buffered path by @sfc-gh-akowalczyk in #862
• SNOW-1475664: Move additional error handling from BufferedTopicPartitionChannel to fix E2E test by @sfc-gh-akowalczyk in #865
• NO_SNOW Add column comments from schemaregistry by @sfc-gh-akowalczyk with contribution from @wornjs in #870
• SNOW-1061854 Run e2e tests with single buffer by @sfc-gh-lshcharbaty in #867
• SNOW-1019628 Enable V2 file cleaner for Snowpipe ingestion by default by @sfc-gh-gjachimko in #868
• Release v2.3.0 by @sfc-gh-mbobowski in #873

New Contributors

• @sfc-gh-krosinski made their first contribution in #824
• @sfc-gh-nsienkiewicz-1 made their first contribution in #842

Full Changelog: v2.2.2...v.2.3.0