Skip to content

v0.5.0
Compare
Choose a tag to compare
@github-actions github-actions released this 09 Aug 09:34
v0.5.0
8e1a114
This commit was signed with the committer’s verified signature.
pan3793 Cheng Pan
GPG key ID: 8001952629BCC75D
Learn about vigilant mode.

Notable Changes

As of 0.5.0, this connector switches from ClickHouse raw gRPC Client to ClickHouse Official Java Client, which brings HTTP protocol support, extending the range of supported versions of ClickHouse Server. In the meanwhile, the gzip, zstd write compression support has been removed, and currently supported codecs are none, lz4(default).

If you upgrade from the previous versions, ONE of the following jars should be used w/ spark-clickhouse-connector-3.3_2.12-0.5.0.jar instead of a single clickhouse-spark-runtime-3.3_2.12-0.4.0.jar.

If you want to connect ClickHouse through gRPC, using

$SPARK_HOME/bin/spark-shell \
  --conf spark.sql.catalog.clickhouse=xenon.clickhouse.ClickHouseCatalog \
  --conf spark.sql.catalog.clickhouse.host=<clickhouse-host> \
  --conf spark.sql.catalog.clickhouse.protocol=grpc \
  --conf spark.sql.catalog.clickhouse.grpc_port=<clickhouse-grpc-port> \
  --conf spark.sql.catalog.clickhouse.user=<username> \
  --conf spark.sql.catalog.clickhouse.password=<password> \
  --conf spark.sql.catalog.clickhouse.database=<default-database> \
  --jars /path/clickhouse-spark-runtime-3.3_2.12:0.5.0.jar,/path/clickhouse-jdbc-0.3.2-patch11-all.jar

and if you prefer to use http, just change

  --conf spark.sql.catalog.clickhouse.protocol=grpc \
  --conf spark.sql.catalog.clickhouse.grpc_port=<clickhouse-grpc-port> \

to

  --conf spark.sql.catalog.clickhouse.protocol=http \
  --conf spark.sql.catalog.clickhouse.http_port=<clickhouse-http-port> \

Change Logs

  • Core: Deserializer consumes InputStream instead of ByteString (#162)
  • Core: Throw ClickHouseException instead of gRPC Exception (#163)
  • Core: Rename CHException (#164)
  • Core: Use ClickHouse Java client
  • Core: Remove gRPC
  • Core: Support compression on reading
  • Core: Simplify deserializeStream (#173)
  • Core: CHException should propagate root cause (#181)
  • Spark: Use ClickHouse Java client
  • Spark: Support compression on reading
  • Spark: Add column comment when create clickhouse table (#176)
  • Spark: Fix reading decimal values (#180)
  • Spark 3.3: Remove zstd support in writing (#166)
  • Spark 3.3: Support write metrics (#169)
  • Docs: Remove gzip compression
  • Docs: Mention HTTP support
  • Docs: Upgrade mkdocs-material
  • Docs: Replace versions w/ variables
  • Docs: Add spark.clickhouse.read.compression.codec
  • Docs: Enrich internal docs
  • Build: Bump clickhouse-jdbc 0.3.2-patch11
  • Build: Bump gradle rat plugin 0.7.1
  • Build: Remove scala-xml version restriction (#175)
  • Build: Algin Jackson version w/ Spark (#177)
  • Build: Bump Gradle 7.5.1
  • Playground: Switch to ClickHouse Java client
  • Playground: Fix dev setup
  • Test: Remove unused SparkClickHouseSingleTestHelper
  • Test: Bump testcontainers-scala 0.40.10 (#168)
Assets 2
Loading