Skip to content

Commit

Permalink
[4.9][clickhouse] Draft clickhouse docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Robert Fekete authored and Robert Fekete committed Nov 9, 2024
1 parent 7195c3e commit f1003c1
Showing 1 changed file with 198 additions and 0 deletions.
198 changes: 198 additions & 0 deletions content/chapter-destinations/clickhouse/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
---
title: ClickHouse database
linktitle: "ClickHouse"
weight: 200
driver: "clickhouse()"
short_description: "Send messages to a ClickHouse database"
---
<!-- This file is under the copyright of Axoflow, and licensed under Apache License 2.0, except for using the Axoflow and AxoSyslog trademarks. -->

Starting with version 4.9.0, {{% param "product_name" %}} can send data to [ClickHouse databases](https://clickhouse.com/) using its [gRPC interface](https://clickhouse.com/docs/en/interfaces/grpc).

## Prerequisites

- A [self-hosted ClickHouse installation](https://clickhouse.com/docs/en/install).

{{% alert title="Warning" color="warning" %}}
ClickHouse Cloud doesn't support the gRPC interface currently.
{{% /alert %}}

- The [gRPC interface](https://clickhouse.com/docs/en/interfaces/grpc) must be enabled in your ClickHouse configuration.
- To configure {{% param "product_name" %}}, you'll need:

- the name of an existing database and a table where you want to send your data, and
- the credentials (username and password) to access the database.

Example configuration:

```sh
destination {
clickhouse(
database("default")
table("demo_table")
user("your-username")
password("your-password")
schema(
"user_id" UInt32 => $R_MSEC,
"message" String => "$MSG",
"timestamp" DateTime => "$R_UNIXTIME",
"metric" Float32 => 3.14
)
);
};
```

## Options

This destination has the following options:

{{< include-headless "chunk/grpc-authentication.md" >}}

{{% include-headless "chunk/option-destination-otlp-batch-bytes.md" %}}

## batch-lines()

| | |
| -------- | ------ |
| Type: | number |
| Default: | 0 |

{{% include-headless "chunk/option-description-destination-batch-lines.md" %}}

{{% include-headless "chunk/option-destination-batch-timeout.md" %}}

{{< include-headless "chunk/option-grpc-channel-args.md" >}}

{{< include-headless "chunk/option-destination-grpc-compression.md" >}}

## database()

| | |
| -------- | -------------------------- |
| Type: | string |
| Default: | `default` |

*Description:* The database where {{% param "product_name" %}} sends the data.

{{< include-headless "chunk/option-destination-diskbuffer.md" >}}

{{< include-headless "chunk/option-destination-frac-digits.md" >}}

{{< include-headless "chunk/option-grpc-headers.md" >}}

{{< include-headless "chunk/option-destination-hook.md" >}}

{{< include-headless "chunk/option-destination-grpc-keep-alive.md" >}}

{{% include-headless "chunk/option-destination-local-timezone.md" %}}

{{% include-headless "chunk/option-destination-log-fifo-size.md" %}}

{{< include-headless "chunk/option-destination-on-error.md" >}}

## password()

| | |
| -------- | ------------ |
| Type: | string |
| Default: | empty string |

*Description:* The password used for authentication.

{{% include-headless "chunk/option-persist-name.md" %}}

## protobuf-schema()

| | |
| -------- | -------------------------- |
| Type: | See the description |
| Default: | - |

*Description:* Sets the schema of the database table from a protobuf schema file.

```shell
protobuf-schema("/tmp/test.proto" => "$MESSAGE", "$PROGRAM", "$HOST", "$PID")
```

An example proto file when using the `protobuf-schema()` option:

```shell
syntax = "proto2";
message CustomRecord {
optional string message = 1;
optional string app = 2;
optional string host = 3;
optional int64 pid = 4;
}
```

Alternatively, you can set the schema with the [`schema()`](#schema) option.

{{% include-headless "chunk/option-destination-retries.md" %}}

## schema()

| | |
| -------- | -------------------------- |
| Type: | arrow list |
| Default: | - |

*Description:* Sets the schema of the database table. On the left side of the arrow, set the name of the column and its type. On the right side, set any {{% param "product_name" %}} template or macro, which gets evaluated on each log that is routed to the destination. For example:

```shell
schema(
"user_id" UInt32 => $R_MSEC,
"message" String => "$MSG",
"timestamp" DateTime => "$R_UNIXTIME",
"metric" Float32 => 3.14
)
```

<!-- FIXME The available column types are: -->

Alternatively, you can set the schema with the [`protobuf-schema()`](#protobuf-schema) option.

{{% include-headless "chunk/option-destination-send-timezone.md" %}}

## table()

| | |
| -------- | -------------------------- |
| Type: | string |
| Default: | - |

*Description:* The name of the table where {{% param "product_name" %}} sends the data.

{{% include-headless "chunk/option-destination-template-escape.md" %}}

{{% include-headless "chunk/option-destination-throttle.md" %}}

{{% include-headless "chunk/option-source-time-reopen.md" %}}

{{% include-headless "chunk/option-destination-timezone.md" %}}

{{< include-headless "chunk/option-destination-ts-format.md" >}}

## url()

| | |
| -------- | -------------------------- |
| Type: | string |
| Default: | `localhost:9100` |

*Description:* The URL of the gRPC receiver.

## user()

| | |
| -------- | ------------ |
| Type: | string |
| Default: | empty string |

*Description:* The username used for authentication.

<a id="worker-partition-key"></a>
{{< include-headless "chunk/option-destination-http-worker-partition-key.md" >}}

{{< include-headless "chunk/option-destination-threaded-workers.md" >}}

0 comments on commit f1003c1

Please sign in to comment.