Skip to content

Commit

Permalink
doc: update EG control-plane metrics design & usage (#3346)
Browse files Browse the repository at this point in the history
* comment polish and doc update

Signed-off-by: shawnh2 <shawnhxh@outlook.com>

* fix doc lint

Signed-off-by: shawnh2 <shawnhxh@outlook.com>

* revert go file changes

Signed-off-by: shawnh2 <shawnhxh@outlook.com>

* add eg cp metrics usage doc

Signed-off-by: shawnh2 <shawnhxh@outlook.com>

* update prometheus metrics usage doc

Signed-off-by: shawnh2 <shawnhxh@outlook.com>

---------

Signed-off-by: shawnh2 <shawnhxh@outlook.com>
  • Loading branch information
shawnh2 authored May 14, 2024
1 parent 675da4d commit 3877aa4
Show file tree
Hide file tree
Showing 8 changed files with 208 additions and 16 deletions.
12 changes: 6 additions & 6 deletions site/content/en/contributions/design/eg-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ title: "Control Plane Observability: Metrics"
This document aims to cover all aspects of envoy gateway control plane metrics observability.

{{% alert title="Note" color="secondary" %}}
**Data plane** observability (while important) is outside of scope for this document. For dataplane observability, refer to [here](../metrics).
**Data plane** observability (while important) is outside of scope for this document. For data plane observability, refer to [here](./proxy-metrics).
{{% /alert %}}

## Current State
Expand Down Expand Up @@ -100,7 +100,7 @@ New APIs will be added to Envoy Gateway config, which are used to manage Control

### EnvoyGatewayTelemetry

``` go
```go
// EnvoyGatewayTelemetry defines telemetry configurations for envoy gateway control plane.
// Control plane will focus on metrics observability telemetry and tracing telemetry later.
type EnvoyGatewayTelemetry struct {
Expand All @@ -113,7 +113,7 @@ type EnvoyGatewayTelemetry struct {

> Prometheus will be exposed on 0.0.0.0:19001, which is not supported to be configured yet.
``` go
```go
// EnvoyGatewayMetrics defines control plane push/pull metrics configurations.
type EnvoyGatewayMetrics struct {
// Sinks defines the metric sinks where metrics are sent to.
Expand Down Expand Up @@ -162,7 +162,7 @@ type EnvoyGatewayPrometheusProvider struct {

+ The following is an example to disable prometheus metric.

``` yaml
```yaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyGateway
gateway:
Expand All @@ -180,7 +180,7 @@ telemetry:
+ The following is an example to send metric via Open Telemetry sink to OTEL gRPC Collector.
``` yaml
```yaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyGateway
gateway:
Expand All @@ -202,7 +202,7 @@ telemetry:
+ The following is an example to disable prometheus metric and send metric via Open Telemetry sink to OTEL HTTP Collector at the same time.
``` yaml
```yaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyGateway
gateway:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
---
title: "Observability: Accesslog"
title: "Data Plane Observability: Accesslog"
---

## Overview

Envoy supports extensible accesslog to different sinks, File, gRPC etc. Envoy supports customizable access log formats using predefined fields as well as arbitrary HTTP request and response headers. Envoy supports several built-in access log filters and extension filters that are registered at runtime.
Envoy supports extensible accesslog to different sinks, File, gRPC etc.

Envoy supports customizable access log formats using predefined fields as well as arbitrary HTTP request and response headers.

Envoy supports several built-in access log filters and extension filters that are registered at runtime.

Envoy Gateway leverages [Gateway API][] for configuring managed Envoy proxies. Gateway API defines core, extended, and implementation-specific API [support levels][] for implementers such as Envoy Gateway to expose features. Since accesslog is not covered by `Core` or `Extended` APIs, EG should provide an easy to config access log formats and sinks per `EnvoyProxy`.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
---
title: "Observability: Metrics"
title: "Data Plane Observability: Metrics"
---

This document aims to cover all aspects of envoy gateway data plane metrics observability.

{{% alert title="Note" color="secondary" %}}
**Control plane** observability (while important) is outside of scope for this document. For control plane observability, refer to [here](./eg-metrics).
{{% /alert %}}

## Overview

Envoy provide robust platform for metrics, Envoy support three different kinds of stats: counter, gauges, histograms.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Observability: Tracing"
title: "Data Plane Observability: Tracing"
---

## Overview
Expand Down
182 changes: 182 additions & 0 deletions site/content/en/latest/tasks/observability/gateway-observability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
---
title: "Gateway Observability"
---

Envoy Gateway provides observability for the ControlPlane and the underlying EnvoyProxy instances.
This task show you how to config gateway control-plane observability, includes metrics.

## Prerequisites

Follow the steps from the [Quickstart](../quickstart) to install Envoy Gateway and the example manifest.
Before proceeding, you should be able to query the example backend using HTTP.

[OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) offers a vendor-agnostic implementation of how to receive, process and export telemetry data.
Install OTel-Collector:

```shell
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm upgrade --install otel-collector open-telemetry/opentelemetry-collector -f https://raw.githubusercontent.com/envoyproxy/gateway/latest/examples/otel-collector/helm-values.yaml -n monitoring --create-namespace --version 0.60.0
```

## Metrics

The default installation of Envoy Gateway installs a default [EnvoyGateway][] configuration and attaches it
using a `ConfigMap`. In this section, we will update this resource to enable various ways to retrieve metrics
from Envoy Gateway.

### Retrieve Prometheus Metrics from Envoy Gateway

By default, prometheus metric is enabled. You can directly retrieve metrics from Envoy Gateway:

```shell
export ENVOY_POD_NAME=$(kubectl get pod -n envoy-gateway-system --selector=control-plane=envoy-gateway,app.kubernetes.io/instance=eg -o jsonpath='{.items[0].metadata.name}')
kubectl port-forward pod/$ENVOY_POD_NAME -n envoy-gateway-system 19001:19001

# check metrics
curl localhost:19001/metrics
```

The following is an example to disable prometheus metric for Envoy Gateway.

{{< tabpane text=true >}}
{{% tab header="Apply from stdin" %}}

```shell
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: envoy-gateway-config
namespace: envoy-gateway-system
data:
envoy-gateway.yaml: |
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyGateway
provider:
type: Kubernetes
gateway:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
telemetry:
metrics:
prometheus:
disable: true
EOF
```

{{% /tab %}}
{{% tab header="Apply from file" %}}
Save and apply the following resource to your cluster:

```yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
name: envoy-gateway-config
namespace: envoy-gateway-system
data:
envoy-gateway.yaml: |
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyGateway
provider:
type: Kubernetes
gateway:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
telemetry:
metrics:
prometheus:
disable: true
```
{{% /tab %}}
{{< /tabpane >}}
After updating the `ConfigMap`, you will need to restart the `envoy-gateway` deployment so the configuration kicks in:

```shell
kubectl rollout restart deployment envoy-gateway -n envoy-gateway-system
```

### Enable Open Telemetry sink in Envoy Gateway

The following is an example to send metric via Open Telemetry sink to OTEL gRPC Collector.

{{< tabpane text=true >}}
{{% tab header="Apply from stdin" %}}

```shell
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: envoy-gateway-config
namespace: envoy-gateway-system
data:
envoy-gateway.yaml: |
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyGateway
provider:
type: Kubernetes
gateway:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
telemetry:
metrics:
sinks:
- type: OpenTelemetry
openTelemetry:
host: otel-collector.monitoring.svc.cluster.local
port: 4317
protocol: grpc
EOF
```

{{% /tab %}}
{{% tab header="Apply from file" %}}
Save and apply the following resource to your cluster:

```yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
name: envoy-gateway-config
namespace: envoy-gateway-system
data:
envoy-gateway.yaml: |
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyGateway
provider:
type: Kubernetes
gateway:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
telemetry:
metrics:
sinks:
- type: OpenTelemetry
openTelemetry:
host: otel-collector.monitoring.svc.cluster.local
port: 4317
protocol: grpc
```

{{% /tab %}}
{{< /tabpane >}}

After updating the `ConfigMap`, you will need to restart the `envoy-gateway` deployment so the configuration kicks in:

```shell
kubectl rollout restart deployment envoy-gateway -n envoy-gateway-system
```

Verify OTel-Collector metrics:

```shell
export OTEL_POD_NAME=$(kubectl get pod -n monitoring --selector=app.kubernetes.io/name=opentelemetry-collector -o jsonpath='{.items[0].metadata.name}')
kubectl port-forward pod/$OTEL_POD_NAME -n monitoring 19001:19001
# check metrics
curl localhost:19001/metrics
```

[EnvoyGateway]: ../../api/extension_types#envoygateway
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ export ENVOY_POD_NAME=$(kubectl get pod -n envoy-gateway-system --selector=gatew
kubectl port-forward pod/$ENVOY_POD_NAME -n envoy-gateway-system 19001:19001

# check metrics
curl localhost:19001/stats/prometheus | grep "default/backend/rule/0/match/0-www"
curl localhost:19001/stats/prometheus | grep "default/backend/rule/0"
```

You can disable metrics by setting the `telemetry.metrics.prometheus.disable` to `true` in the `EnvoyProxy` CRD.
Expand All @@ -82,7 +82,7 @@ export OTEL_POD_NAME=$(kubectl get pod -n monitoring --selector=app.kubernetes.i
kubectl port-forward pod/$OTEL_POD_NAME -n monitoring 19001:19001

# check metrics
curl localhost:19001/metrics | grep "default/backend/rule/0/match/0-www"
curl localhost:19001/metrics | grep "default/backend/rule/0"
```

## Logs
Expand Down
Loading

0 comments on commit 3877aa4

Please sign in to comment.