From 9750ada0b8683157c53cf063cf3fb221430a0c06 Mon Sep 17 00:00:00 2001 From: zirain Date: Wed, 3 Jul 2024 18:30:27 +0800 Subject: [PATCH] docs: CEL Expressions for Access Logging Signed-off-by: zirain --- .../tasks/observability/proxy-accesslog.md | 175 ++++++++++++++++++ .../observability/proxy-observability.md | 50 +---- 2 files changed, 176 insertions(+), 49 deletions(-) create mode 100644 site/content/en/latest/tasks/observability/proxy-accesslog.md diff --git a/site/content/en/latest/tasks/observability/proxy-accesslog.md b/site/content/en/latest/tasks/observability/proxy-accesslog.md new file mode 100644 index 00000000000..5163bee56f4 --- /dev/null +++ b/site/content/en/latest/tasks/observability/proxy-accesslog.md @@ -0,0 +1,175 @@ +--- +title: "Proxy Access Log" +--- + +Envoy Gateway provides observability for the ControlPlane and the underlying EnvoyProxy instances. +This task show you how to config proxy access logs. + +## Prerequisites + +{{< boilerplate o11y_prerequisites >}} + +By default, the Service type of `loki` is ClusterIP, you can change it to LoadBalancer type for further usage: + +```shell +kubectl patch service loki -n monitoring -p '{"spec": {"type": "LoadBalancer"}}' +``` + +Expose endpoints: + +```shell +LOKI_IP=$(kubectl get svc loki -n monitoring -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +``` + +## Default Access Log + +If custom format string is not specified, Envoy Gateway uses the following default format: + +```json +{"start_time":"%START_TIME%","method":"%REQ(:METHOD)%","x-envoy-origin-path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","protocol":"%PROTOCOL%","response_code":"%RESPONSE_CODE%","response_flags":"%RESPONSE_FLAGS%","response_code_details":"%RESPONSE_CODE_DETAILS%","connection_termination_details":"%CONNECTION_TERMINATION_DETAILS%","upstream_transport_failure_reason":"%UPSTREAM_TRANSPORT_FAILURE_REASON%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","duration":"%DURATION%","x-envoy-upstream-service-time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","x-forwarded-for":"%REQ(X-FORWARDED-FOR)%","user-agent":"%REQ(USER-AGENT)%","x-request-id":"%REQ(X-REQUEST-ID)%",":authority":"%REQ(:AUTHORITY)%","upstream_host":"%UPSTREAM_HOST%","upstream_cluster":"%UPSTREAM_CLUSTER%","upstream_local_address":"%UPSTREAM_LOCAL_ADDRESS%","downstream_local_address":"%DOWNSTREAM_LOCAL_ADDRESS%","downstream_remote_address":"%DOWNSTREAM_REMOTE_ADDRESS%","requested_server_name":"%REQUESTED_SERVER_NAME%","route_name":"%ROUTE_NAME%"} +``` + +> Note: Envoy Gateway disable envoy headers by default, you can enable it by setting `EnableEnvoyHeaders` to `true` in the [ClientTrafficPolicy](../../api/extension_types#backendtrafficpolicy) CRD. + + +Verify logs from loki: + +```shell +curl -s "http://$LOKI_IP:3100/loki/api/v1/query_range" --data-urlencode "query={job=\"fluentbit\"}" | jq '.data.result[0].values' +``` + +## Disable Access Log + +If you want to disable it, set the `telemetry.accesslog.disable` to `true` in the `EnvoyProxy` CRD. + +```shell +kubectl apply -f - <}} -By default, the Service type of `loki` is ClusterIP, you can change it to LoadBalancer type for further usage: - -```shell -kubectl patch service loki -n monitoring -p '{"spec": {"type": "LoadBalancer"}}' -``` - Expose endpoints: ```shell -LOKI_IP=$(kubectl get svc loki -n monitoring -o jsonpath='{.status.loadBalancer.ingress[0].ip}') TEMPO_IP=$(kubectl get svc tempo -n monitoring -o jsonpath='{.status.loadBalancer.ingress[0].ip}') ``` @@ -59,47 +52,6 @@ kubectl port-forward pod/$OTEL_POD_NAME -n monitoring 19001:19001 curl localhost:19001/metrics | grep "default/backend/rule/0" ``` -## Logs - -If custom format string is not specified, Envoy Gateway uses the following default format: - -```json -{"start_time":"%START_TIME%","method":"%REQ(:METHOD)%","x-envoy-origin-path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","protocol":"%PROTOCOL%","response_code":"%RESPONSE_CODE%","response_flags":"%RESPONSE_FLAGS%","response_code_details":"%RESPONSE_CODE_DETAILS%","connection_termination_details":"%CONNECTION_TERMINATION_DETAILS%","upstream_transport_failure_reason":"%UPSTREAM_TRANSPORT_FAILURE_REASON%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","duration":"%DURATION%","x-envoy-upstream-service-time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","x-forwarded-for":"%REQ(X-FORWARDED-FOR)%","user-agent":"%REQ(USER-AGENT)%","x-request-id":"%REQ(X-REQUEST-ID)%",":authority":"%REQ(:AUTHORITY)%","upstream_host":"%UPSTREAM_HOST%","upstream_cluster":"%UPSTREAM_CLUSTER%","upstream_local_address":"%UPSTREAM_LOCAL_ADDRESS%","downstream_local_address":"%DOWNSTREAM_LOCAL_ADDRESS%","downstream_remote_address":"%DOWNSTREAM_REMOTE_ADDRESS%","requested_server_name":"%REQUESTED_SERVER_NAME%","route_name":"%ROUTE_NAME%"} -``` - -> Note: Envoy Gateway disable envoy headers by default, you can enable it by setting `EnableEnvoyHeaders` to `true` in the [ClientTrafficPolicy](../../api/extension_types#backendtrafficpolicy) CRD. - - -Verify logs from loki: - -```shell -curl -s "http://$LOKI_IP:3100/loki/api/v1/query_range" --data-urlencode "query={job=\"fluentbit\"}" | jq '.data.result[0].values' -``` - -If you want to disable it, set the `telemetry.accesslog.disable` to `true` in the `EnvoyProxy` CRD. - -```shell -kubectl apply -f https://raw.githubusercontent.com/envoyproxy/gateway/latest/examples/kubernetes/accesslog/disable-accesslog.yaml -``` - -Envoy Gateway can send logs to OpenTelemetry Sink. - -```shell -kubectl apply -f https://raw.githubusercontent.com/envoyproxy/gateway/latest/examples/kubernetes/accesslog/otel-accesslog.yaml -``` - -Verify logs from loki: - -```shell -curl -s "http://$LOKI_IP:3100/loki/api/v1/query_range" --data-urlencode "query={exporter=\"OTLP\"}" | jq '.data.result[0].values' -``` - -Envoy Gateway provides additional metadata about the K8s resources that were translated to certain envoy resources. -For example, details about the `HTTPRoute` and `GRPCRoute` (kind, group, name, namespace and annotations) are available -for access log formatter using the `METADATA` operator. To enrich logs, users can add log operator such as: -`%METADATA(ROUTE:envoy-gateway:resources)%` to their access log format. - - ## Traces By default, Envoy Gateway doesn't send traces to OpenTelemetry Sink.