1.6.0: Migrated to `mvn` build

Summary

In this release, we migrated from Scala sbt based build system to Java maven build system. This change fully enables us to use our tooling infrastructure such as GitHub workflows, dependency checks etc. We also updated all dependencies to their latest versions.

Refactoring

• #69: Migrated from sbt to maven build system.

Dependency Updates

Compile Dependency Updates

• Added com.exasol:error-reporting-java:1.0.1
• Added com.exasol:import-export-udf-common-scala_2.13:1.1.1
• Added com.google.guava:guava:32.1.1-jre
• Added io.confluent:kafka-avro-serializer:7.4.1
• Added org.scala-lang.modules:scala-collection-compat_2.13:2.11.0
• Added org.scala-lang:scala-library:2.13.3

Test Dependency Updates

• Added com.exasol:exasol-testcontainers:6.6.1
• Added com.exasol:hamcrest-resultset-matcher:1.6.0
• Added com.exasol:test-db-builder-java:3.4.2
• Added com.sksamuel.avro4s:avro4s-core_2.13:4.1.1
• Added io.confluent:kafka-streams-avro-serde:7.4.1
• Added io.github.embeddedkafka:embedded-kafka-schema-registry_2.13:7.4.1
• Added io.netty:netty-handler:4.1.95.Final
• Added org.mockito:mockito-core:5.4.0
• Added org.scalatestplus:scalatestplus-mockito_2.13:1.0.0-M2
• Added org.scalatest:scalatest_2.13:3.2.16

Plugin Dependency Updates

• Added com.diffplug.spotless:spotless-maven-plugin:2.37.0
• Added com.exasol:artifact-reference-checker-maven-plugin:0.4.2
• Added com.exasol:error-code-crawler-maven-plugin:1.3.0
• Added com.exasol:project-keeper-maven-plugin:2.9.9
• Added io.github.evis:scalafix-maven-plugin_2.13:0.1.8_0.11.0
• Added io.github.zlika:reproducible-build-maven-plugin:0.16
• Added net.alchim31.maven:scala-maven-plugin:4.8.1
• Added org.apache.maven.plugins:maven-assembly-plugin:3.6.0
• Added org.apache.maven.plugins:maven-clean-plugin:2.5
• Added org.apache.maven.plugins:maven-compiler-plugin:3.11.0
• Added org.apache.maven.plugins:maven-deploy-plugin:2.7
• Added org.apache.maven.plugins:maven-enforcer-plugin:3.3.0
• Added org.apache.maven.plugins:maven-failsafe-plugin:3.1.2
• Added org.apache.maven.plugins:maven-install-plugin:2.4
• Added org.apache.maven.plugins:maven-jar-plugin:3.3.0
• Added org.apache.maven.plugins:maven-javadoc-plugin:3.5.0
• Added org.apache.maven.plugins:maven-resources-plugin:2.6
• Added org.apache.maven.plugins:maven-site-plugin:3.3
• Added org.apache.maven.plugins:maven-surefire-plugin:3.1.2
• Added org.basepom.maven:duplicate-finder-maven-plugin:2.0.1
• Added org.codehaus.mojo:flatten-maven-plugin:1.5.0
• Added org.codehaus.mojo:versions-maven-plugin:2.16.0
• Added org.itsallcode:openfasttrace-maven-plugin:1.6.2
• Added org.jacoco:jacoco-maven-plugin:0.8.10
• Added org.scalastyle:scalastyle-maven-plugin:1.0.0
• Added org.scalatest:scalatest-maven-plugin:2.2.0
• Added org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184
• Added org.sonatype.ossindex.maven:ossindex-maven-plugin:3.2.0

Fixed mismatch between uploaded jar and checksum

Summary

In this release, we fixed mismatch between release artifact jar file and uploaded checksum file value.

Bug Fixes

• #67: Fixed mismatch between jar artifact and checksum value

Dependency Updates

Runtime Dependency Updates

• Updated org.scala-lang:scala-library:2.13.7 to 2.13.8

Test Dependency Updates

• Updated com.exasol:test-db-builder-java:3.2.1 to 3.2.2

Plugin Updates

• Updated org.scalameta:sbt-scalafmt:2.4.5 to 2.4.6
• Updated org.scoverage:sbt-scoverage:1.9.2 to 1.9.3
• Updated com.timushev.sbt:sbt-updates:0.6.0 to 0.6.1

Fixed Log4J vulnerabilities and updated dependencies

Summary

This release fixes Log4J vulnerabilities and updates dependencies.

Bug Fixes

• #64: Fixed Log4J vulnerabilities and updated dependencies

Dependency Updates

Runtime Dependency Updates

• Updated com.exasol:error-reporting-java:0.4.0 to 0.4.1
• Updated org.apache.kafka:kafka-clients:3.0.0 to 7.0.1-ce
• Updated io.confluent:kafka-avro-serializer:6.2.1 to 7.0.1
• Updated org.scala-lang.modules:scala-collection-compat_2.13:2.5.0 to 2.6.0

Test Dependency Updates

• Updated org.mockito:mockito-core:4.0.0 to 4.2.0
• Updated io.github.embeddedkafka:embedded-kafka-schema-registry_2.13:6.2.1 to 7.0.0
• Updated io.confluent:kafka-streams-avro-serde:6.2.1 to 7.0.1
• Updated com.sksamuel.avro4s:avro4s-core_2.13:4.0.11 to 4.0.12

Plugin Updates

• Updated org.scalameta:sbt-scalafmt:2.4.3 to 2.4.5
• Updated org.scoverage:sbt-scoverage:1.9.0 to 1.9.1
• Updated org.wartremover:sbt-wartremover-contrib:1.3.12 to 1.3.13

Fixed setting SSL keystore and truststore files

Summary

This release fixes a bug that skips setting SSL keystore and truststore files when using SASL_SSL protocol.

Bug Fixes

• #61: Fixed bug that skips setting SSL files when using SASL_SSL protocol

Dependency Updates

Runtime Dependency Updates

Test Dependency Updates

• Updated com.exasol:exasol-testcontainers:5.1.0 to 5.1.1

Plugin Updates

Azure Event Hubs Support

Summary

In this release, we added documentation for using the connector to import data from Azure Event Hubs. Please check out the user guide for more information.

Bug Fixes

• #54: Fixed issue with setting empty values to property keys

Documentation

• #56: Updated user guide with Azure Event Hubs usage information

Refactoring

• #55: Added optional check for keystore or truststore files when using SASL_SSL protocol

Dependency Updates

Runtime Dependency Updates

• Updated io.confluent:kafka-avro-serializer:6.2.0 to 6.2.1
• Updated org.apache.kafka:kafka-clients:2.8.0 to 3.0.0

Test Dependency Updates

• Updated org.scalatest:scalatest:3.2.9 to 3.2.10
• Updated org.mockito:mockito-core:3.12.4 to 4.0.0
• Updated com.exasol:exasol-testcontainers:4.0.1 to 5.1.0
• Updated com.exasol:hamcrest-resultset-matcher:1.4.1 to 1.5.1
• Updated com.sksamuel.avro4s:avro4s-core:4.0.10 to 4.0.11
• Updated io.confluent:kafka-streams-avro-serde:6.2.0 to 6.2.1
• Updated io.github.embeddedkafka:embedded-kafka-schema-registry:6.2.0 to 6.2.1

Plugin Updates

• Updated org.scoverage:sbt-scoverage:1.8.2 to 1.9.0

Automatic Timestamp Converter

Summary

In this release, we added feature to convert long values to timestamps since epoch. In addition, we improved exceptions by adding our unified error codes.

Features

• #51: Added timestamp converter for long values

Refactorings

• #12: Added unified error codes

Dependency Updates

Runtime Dependency Updates

Test Dependency Updates

• Updated com.exasol:exasol-testcontainers:4.0.0 to 4.0.1
• Updated org.mockito:mockito-core:3.11.2 to 3.12.4
• Added com.sksamuel.avro4s:avro4s-core:4.0.10
• Added io.confluent:kafka-streams-avro-serde:6.2.0
• Removed org.testcontainers:kafka:1.16.0

Plugin Updates

• Updated com.eed3si9n:sbt-assembly:1.0.0 to 1.1.0
• Updated net.bzzt:sbt-reproducible-builds:0.28 to 0.30

Support SASL Authentication

Summary

This release adds support for SASL authentication. Importer client can choose SASL mechanism (PLAIN, DIGEST-*, SCRAM-*) and provide JAAS configuration for authentication. Please check the user guide for more information.

Features

• Added support for SASL_PLAINTEXT and SASL_SSL security protocols (PR #47)

Runtime Dependency Updates

Test Dependency Updates

• Updated com.exasol:exasol-testcontainers:3.5.3 to 4.0.0
• Updated com.exasol:hamcrest-resultset-matcher:1.4.0 to 1.4.1
• Updated com.exasol:test-db-builder-java:3.2.0 to 3.2.1

Plugin Updates

• Updated com.timushev.sbt:sbt-updates:0.5.3 to 0.6.0

Bug Fixes and Refactorings

Summary

In this release we fixed a bug that occurs when Kafka resets offsets while user consuming all records until the latest offset. We also migrated the continuous integration (CI) environment to Github actions.

Bug Fixes

• #41: Fixed failing consumer when Kafka topic offset resets

Refactoring

• #44: Migrated to Github Actions build infrastructure

Runtime Dependency Updates

• Updated org.scala-lang.modules:scala-collection-compat:2.4.4 to 2.5.0

Test Dependency Updates

• Updated org.testcontainers:kafka:test:1.15.3 to 1.16.0

Plugin Updates

• Added org.scalameta:sbt-scalafmt:2.4.3
• Updated org.wartremover:sbt-wartremover:2.4.15 to 2.4.16
• Removed com.lucidchart:sbt-scalafmt-coursier:1.16
• Removed com.typesafe.sbt:sbt-git:1.0.1
• Removed com.github.cb372:sbt-explicit-dependencies:0.2.16
• Updated org.wartremover:sbt-wartremover:2.4.15 to 2.4.16

Fixed bug in consuming all offsets

Summary

In this release, we fixed a bug that can go into infinite waiting loop if polled records from Kafka are empty.

Bugfixes

• #39: Fixed bug related to consuming all offsets when already at end of partition

Runtime Dependency Updates

• Updated io.confluent:kafka-avro-serializer:6.1.1 to 6.2.0

Test Dependency Updates

• Added com.exasol:test-db-builder-java:3.2.0
• Added com.exasol:exasol-testcontainers:3.5.3
• Added com.exasol:hamcrest-resultset-matcher:1.4.0
• Added org.testcontainers:kafka:1.15.3
• Updated org.mockito:mockito-core:3.11.0 to 3.11.2
• Updated io.github.embeddedkafka:embedded-kafka-schema-registry:6.1.1 to 6.2.0

Plugin Updates

• Updated org.scoverage:sbt-coveralls:1.2.7 to 1.3.1
• Updated net.bzzt:sbt-reproducible-builds:0.25 to 0.28

Support for record fields selection and consuming all offset records

Summary

The release 1.1.0 added support for accessing fields of Kafka record, added an option to consume records until the last offset of partition. In addition, we fixed bugs related to null record values, missing properties and case sensitive parameters.

Features

• #28: Added support to access all fields in Kafka record
• #32: Added option to read until last offset of a partition

Bugfixes

• #29: Fixed issues with record null values in a topic
• #30: Fixed bug with missing properties in connection object
• #35: Fixed bug with case sensitivity in record format parameters

Runtime Dependency Updates

• Updated sbt.version:1.4.4 to 1.5.3
• Updated Scala version 2.13.5 to 2.13.6
• Updated org.apache.kafka:kafka-clients:2.6.0 to 2.8.0
• Updated org.scala-lang.modules:scala-collection-compat:2.4.3 to 2.4.4

Test Dependency Updates

• Updated org.scalatest:scalatest:3.2.8 to 3.2.9
• Updated org.mockito:mockito-core:3.9.0 to 3.11.0

Plugin Updates

• Updated com.eed3si9n:sbt-assembly:0.15.0 to 1.0.0
• Updated com.typesafe.sbt:sbt-git:1.0.0 to 1.0.1
• Updated org.scoverage:sbt-scoverage:1.7.3 to 1.8.2
• Updated org.wartremover:sbt-wartremover:2.4.13 to 2.4.15
• Updated org.wartremover:sbt-wartremover-contib:1.3.11 to 1.3.12