grpc: add clickhouse dest

[alltilla]

Example config:

log {
  source { example-msg-generator(); };
  destination {
    clickhouse(
      database("default")
      table("my_first_table")
      user("default")
      password("pw")
      schema(
        "user_id" UInt32 => $R_MSEC,
        "message" String => "$MSG",
        "timestamp" DateTime => "$R_UNIXTIME",
        "metric" Float32 => 3.14
      )
      workers(4)
      batch-lines(1000)
      batch-timeout(1000)
    );
  };
};

table creation in clickhouse:

CREATE TABLE my_first_table
(
    user_id UInt32,
    message String,
    timestamp DateTime,
    metric Float32
)
ENGINE = MergeTree
PRIMARY KEY (user_id, timestamp)

make sure to enable grpc in clickhouse:

/etc/clickhouse-server/config.d# cat grpc.xml 
<clickhouse>
        <grpc_port>9100</grpc_port>
        <grpc>
                <enable_ssl>false</enable_ssl>

                <!-- The following two files are used only if SSL is enabled -->
                <ssl_cert_file>/path/to/ssl_cert_file</ssl_cert_file>
                <ssl_key_file>/path/to/ssl_key_file</ssl_key_file>

                <!-- Whether server requests client for a certificate -->
                <ssl_require_client_auth>false</ssl_require_client_auth>

                <!-- The following file is used only if ssl_require_client_auth=true -->
                <ssl_ca_cert_file>/path/to/ssl_ca_cert_file</ssl_ca_cert_file>

                <!-- Default compression algorithm (applied if client doesn't specify another algorithm, see result_compression in QueryInfo).
                     Supported algorithms: none, deflate, gzip, stream_gzip -->
                <compression>deflate</compression>

                <!-- Default compression level (applied if client doesn't specify another level, see result_compression in QueryInfo).
                     Supported levels: none, low, medium, high -->
                <compression_level>medium</compression_level>

                <!-- Send/receive message size limits in bytes. -1 means unlimited -->
                <max_send_message_size>-1</max_send_message_size>
                <max_receive_message_size>-1</max_receive_message_size>

                <!-- Enable if you want to get detailed logs -->
                <verbose_logs>false</verbose_logs>
        </grpc>
</clickhouse>

---

For reviewers:

• I think the column <-> type mapping needs at least a big manual testing round. I am not sure I did everything correctly.

Depends on #338

[github-actions]

This Pull Request introduces config grammar changes

axoflow/9aa7142e84e7fe4ae29c65cc922fa2f9835f2516 -> alltilla/grpc-clickhouse

--- a/destination
+++ b/destination

+clickhouse(
+    auth(
+        adc(<empty>)
+        alts(
+            <empty>
+            target-service-accounts(
+                <empty>
+                <string>
+            )
+        )
+        insecure(<empty>)
+        tls(
+            <empty>
+            ca-file(<string>)
+            cert-file(<string>)
+            key-file(<string>)
+        )
+    )
+    batch-bytes(<positive-integer>)
+    batch-lines(<nonnegative-integer>)
+    batch-timeout(<positive-integer>)
+    channel-args(
+        <empty>
+        <string> => <number>
+        <string> => <string>
+    )
+    compression(<yesno>)
+    database(<string>)
+    disk-buffer(
+        capacity-bytes(<number>)
+        compaction(<yesno>)
+        dir(<string>)
+        disk-buf-size(<number>)
+        flow-control-window-bytes(<nonnegative-integer>)
+        flow-control-window-size(<nonnegative-integer>)
+        front-cache-size(<nonnegative-integer>)
+        mem-buf-length(<nonnegative-integer>)
+        mem-buf-size(<nonnegative-integer>)
+        prealloc(<yesno>)
+        qout-size(<nonnegative-integer>)
+        reliable(<yesno>)
+        truncate-size-ratio(<nonnegative-float>)
+    )
+    flags(
+        <empty>
+        <string>
+    )
+    frac-digits(<nonnegative-integer>)
+    headers(
+        <empty>
+        <string> => <template-content>
+        <string> => <template-reference>
+    )
+    hook-commands(
+        setup(<string>)
+        shutdown(<string>)
+        startup(<string>)
+        teardown(<string>)
+    )
+    internal(<yesno>)
+    keep-alive(
+        <empty>
+        max-pings-without-data(<nonnegative-integer>)
+        time(<nonnegative-integer>)
+        timeout(<nonnegative-integer>)
+    )
+    local-time-zone(<string>)
+    log-fifo-size(<positive-integer>)
+    on-error(<string>)
+    password(<string>)
+    persist-name(<string>)
+    protobuf-schema(<path> => <template-content-list>)
+    retries(<positive-integer>)
+    schema(
+        <empty>
+        <string> <string> => <template-content>
+        <string> => <template-content>
+    )
+    send-time-zone(<string>)
+    table(<string>)
+    template-escape(<yesno>)
+    throttle(<nonnegative-integer>)
+    time-reopen(<positive-integer>)
+    time-zone(<string>)
+    ts-format(<string>)
+    url(<string>)
+    user(<string>)
+    worker-partition-key(<template-content>)
+    workers(<positive-integer>)
+)

[alltilla]

rebased to main and fixed the copyright check

[OverOrion]

Overall LGTM, nice!

[OverOrion]

LGTM, nice! 🚀