-
Notifications
You must be signed in to change notification settings - Fork 360
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: CEL Expressions for Access Logging
Signed-off-by: zirain <zirain2009@gmail.com>
- Loading branch information
Showing
2 changed files
with
176 additions
and
49 deletions.
There are no files selected for viewing
175 changes: 175 additions & 0 deletions
175
site/content/en/latest/tasks/observability/proxy-accesslog.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,175 @@ | ||
| --- | ||
| title: "Proxy Access Log" | ||
| --- | ||
|
|
||
| Envoy Gateway provides observability for the ControlPlane and the underlying EnvoyProxy instances. | ||
| This task show you how to config proxy access logs. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| {{< boilerplate o11y_prerequisites >}} | ||
|
|
||
| By default, the Service type of `loki` is ClusterIP, you can change it to LoadBalancer type for further usage: | ||
|
|
||
| ```shell | ||
| kubectl patch service loki -n monitoring -p '{"spec": {"type": "LoadBalancer"}}' | ||
| ``` | ||
|
|
||
| Expose endpoints: | ||
|
|
||
| ```shell | ||
| LOKI_IP=$(kubectl get svc loki -n monitoring -o jsonpath='{.status.loadBalancer.ingress[0].ip}') | ||
| ``` | ||
|
|
||
| ## Default Access Log | ||
|
|
||
| If custom format string is not specified, Envoy Gateway uses the following default format: | ||
|
|
||
| ```json | ||
| {"start_time":"%START_TIME%","method":"%REQ(:METHOD)%","x-envoy-origin-path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","protocol":"%PROTOCOL%","response_code":"%RESPONSE_CODE%","response_flags":"%RESPONSE_FLAGS%","response_code_details":"%RESPONSE_CODE_DETAILS%","connection_termination_details":"%CONNECTION_TERMINATION_DETAILS%","upstream_transport_failure_reason":"%UPSTREAM_TRANSPORT_FAILURE_REASON%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","duration":"%DURATION%","x-envoy-upstream-service-time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","x-forwarded-for":"%REQ(X-FORWARDED-FOR)%","user-agent":"%REQ(USER-AGENT)%","x-request-id":"%REQ(X-REQUEST-ID)%",":authority":"%REQ(:AUTHORITY)%","upstream_host":"%UPSTREAM_HOST%","upstream_cluster":"%UPSTREAM_CLUSTER%","upstream_local_address":"%UPSTREAM_LOCAL_ADDRESS%","downstream_local_address":"%DOWNSTREAM_LOCAL_ADDRESS%","downstream_remote_address":"%DOWNSTREAM_REMOTE_ADDRESS%","requested_server_name":"%REQUESTED_SERVER_NAME%","route_name":"%ROUTE_NAME%"} | ||
| ``` | ||
|
|
||
| > Note: Envoy Gateway disable envoy headers by default, you can enable it by setting `EnableEnvoyHeaders` to `true` in the [ClientTrafficPolicy](../../api/extension_types#backendtrafficpolicy) CRD. | ||
|
|
||
| Verify logs from loki: | ||
|
|
||
| ```shell | ||
| curl -s "http://$LOKI_IP:3100/loki/api/v1/query_range" --data-urlencode "query={job=\"fluentbit\"}" | jq '.data.result[0].values' | ||
| ``` | ||
|
|
||
| ## Disable Access Log | ||
|
|
||
| If you want to disable it, set the `telemetry.accesslog.disable` to `true` in the `EnvoyProxy` CRD. | ||
|
|
||
| ```shell | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: GatewayClass | ||
| metadata: | ||
| name: eg | ||
| spec: | ||
| controllerName: gateway.envoyproxy.io/gatewayclass-controller | ||
| parametersRef: | ||
| group: gateway.envoyproxy.io | ||
| kind: EnvoyProxy | ||
| name: disable-accesslog | ||
| namespace: envoy-gateway-system | ||
| --- | ||
| apiVersion: gateway.envoyproxy.io/v1alpha1 | ||
| kind: EnvoyProxy | ||
| metadata: | ||
| name: disable-accesslog | ||
| namespace: envoy-gateway-system | ||
| spec: | ||
| telemetry: | ||
| accessLog: | ||
| disable: true | ||
| EOF | ||
| ``` | ||
|
|
||
| ## OpenTelemetry Sink | ||
|
|
||
| Envoy Gateway can send logs to OpenTelemetry Sink. | ||
|
|
||
| ```shell | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: GatewayClass | ||
| metadata: | ||
| name: eg | ||
| spec: | ||
| controllerName: gateway.envoyproxy.io/gatewayclass-controller | ||
| parametersRef: | ||
| group: gateway.envoyproxy.io | ||
| kind: EnvoyProxy | ||
| name: otel-access-logging | ||
| namespace: envoy-gateway-system | ||
| --- | ||
| apiVersion: gateway.envoyproxy.io/v1alpha1 | ||
| kind: EnvoyProxy | ||
| metadata: | ||
| name: otel-access-logging | ||
| namespace: envoy-gateway-system | ||
| spec: | ||
| telemetry: | ||
| accessLog: | ||
| settings: | ||
| - format: | ||
| type: Text | ||
| text: | | ||
| [%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%" | ||
| sinks: | ||
| - type: OpenTelemetry | ||
| openTelemetry: | ||
| host: otel-collector.monitoring.svc.cluster.local | ||
| port: 4317 | ||
| resources: | ||
| k8s.cluster.name: "cluster-1" | ||
| EOF | ||
| ``` | ||
|
|
||
| Verify logs from loki: | ||
|
|
||
| ```shell | ||
| curl -s "http://$LOKI_IP:3100/loki/api/v1/query_range" --data-urlencode "query={exporter=\"OTLP\"}" | jq '.data.result[0].values' | ||
| ``` | ||
|
|
||
| ## CEL Expressions | ||
|
|
||
| Envoy Gateway provides [CEL expressions](https://www.envoyproxy.io/docs/envoy/latest/xds/type/v3/cel.proto.html#common-expression-language-cel-proto) to filter access log . | ||
|
|
||
| For example, you can use the expression `'x-envoy-logged' in request.headers` to filter logs that contain the `x-envoy-logged` header. | ||
|
|
||
| ```shell | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: GatewayClass | ||
| metadata: | ||
| name: eg | ||
| spec: | ||
| controllerName: gateway.envoyproxy.io/gatewayclass-controller | ||
| parametersRef: | ||
| group: gateway.envoyproxy.io | ||
| kind: EnvoyProxy | ||
| name: otel-access-logging | ||
| namespace: envoy-gateway-system | ||
| --- | ||
| apiVersion: gateway.envoyproxy.io/v1alpha1 | ||
| kind: EnvoyProxy | ||
| metadata: | ||
| name: otel-access-logging | ||
| namespace: envoy-gateway-system | ||
| spec: | ||
| telemetry: | ||
| accessLog: | ||
| settings: | ||
| - format: | ||
| type: Text | ||
| text: | | ||
| [%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%" | ||
| matches: | ||
| - "'x-envoy-logged' in request.headers" | ||
| sinks: | ||
| - type: OpenTelemetry | ||
| openTelemetry: | ||
| host: otel-collector.monitoring.svc.cluster.local | ||
| port: 4317 | ||
| resources: | ||
| k8s.cluster.name: "cluster-1" | ||
| EOF | ||
| ``` | ||
|
|
||
| Verify logs from loki: | ||
|
|
||
| ```shell | ||
| curl -s "http://$LOKI_IP:3100/loki/api/v1/query_range" --data-urlencode "query={exporter=\"OTLP\"}" | jq '.data.result[0].values' | ||
| ``` | ||
|
|
||
|
|
||
| ### Additional Metadata | ||
|
|
||
| Envoy Gateway provides additional metadata about the K8s resources that were translated to certain envoy resources. | ||
| For example, details about the `HTTPRoute` and `GRPCRoute` (kind, group, name, namespace and annotations) are available | ||
| for access log formatter using the `METADATA` operator. To enrich logs, users can add log operator such as: | ||
| `%METADATA(ROUTE:envoy-gateway:resources)%` to their access log format. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters