diff --git a/site/content/en/contributions/design/eg-metrics.md b/site/content/en/contributions/design/eg-metrics.md index 0ca0e7229ec..6b27bbac756 100644 --- a/site/content/en/contributions/design/eg-metrics.md +++ b/site/content/en/contributions/design/eg-metrics.md @@ -6,7 +6,7 @@ title: "Control Plane Observability: Metrics" This document aims to cover all aspects of envoy gateway control plane metrics observability. {{% alert title="Note" color="secondary" %}} -**Data plane** observability (while important) is outside of scope for this document. For dataplane observability, refer to [here](../metrics). +**Data plane** observability (while important) is outside of scope for this document. For data plane observability, refer to [here](./proxy-metrics). {{% /alert %}} ## Current State @@ -100,7 +100,7 @@ New APIs will be added to Envoy Gateway config, which are used to manage Control ### EnvoyGatewayTelemetry -``` go +```go // EnvoyGatewayTelemetry defines telemetry configurations for envoy gateway control plane. // Control plane will focus on metrics observability telemetry and tracing telemetry later. type EnvoyGatewayTelemetry struct { @@ -113,7 +113,7 @@ type EnvoyGatewayTelemetry struct { > Prometheus will be exposed on 0.0.0.0:19001, which is not supported to be configured yet. -``` go +```go // EnvoyGatewayMetrics defines control plane push/pull metrics configurations. type EnvoyGatewayMetrics struct { // Sinks defines the metric sinks where metrics are sent to. @@ -162,7 +162,7 @@ type EnvoyGatewayPrometheusProvider struct { + The following is an example to disable prometheus metric. -``` yaml +```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyGateway gateway: @@ -180,7 +180,7 @@ telemetry: + The following is an example to send metric via Open Telemetry sink to OTEL gRPC Collector. -``` yaml +```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyGateway gateway: @@ -202,7 +202,7 @@ telemetry: + The following is an example to disable prometheus metric and send metric via Open Telemetry sink to OTEL HTTP Collector at the same time. -``` yaml +```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyGateway gateway: diff --git a/site/content/en/contributions/design/accesslog.md b/site/content/en/contributions/design/proxy-accesslog.md similarity index 96% rename from site/content/en/contributions/design/accesslog.md rename to site/content/en/contributions/design/proxy-accesslog.md index a229d5f6eff..7557e97ad2a 100644 --- a/site/content/en/contributions/design/accesslog.md +++ b/site/content/en/contributions/design/proxy-accesslog.md @@ -1,10 +1,14 @@ --- -title: "Observability: Accesslog" +title: "Data Plane Observability: Accesslog" --- ## Overview -Envoy supports extensible accesslog to different sinks, File, gRPC etc. Envoy supports customizable access log formats using predefined fields as well as arbitrary HTTP request and response headers. Envoy supports several built-in access log filters and extension filters that are registered at runtime. +Envoy supports extensible accesslog to different sinks, File, gRPC etc. + +Envoy supports customizable access log formats using predefined fields as well as arbitrary HTTP request and response headers. + +Envoy supports several built-in access log filters and extension filters that are registered at runtime. Envoy Gateway leverages [Gateway API][] for configuring managed Envoy proxies. Gateway API defines core, extended, and implementation-specific API [support levels][] for implementers such as Envoy Gateway to expose features. Since accesslog is not covered by `Core` or `Extended` APIs, EG should provide an easy to config access log formats and sinks per `EnvoyProxy`. diff --git a/site/content/en/contributions/design/metrics.md b/site/content/en/contributions/design/proxy-metrics.md similarity index 91% rename from site/content/en/contributions/design/metrics.md rename to site/content/en/contributions/design/proxy-metrics.md index 78b05eea98e..7ae2f7b30ef 100644 --- a/site/content/en/contributions/design/metrics.md +++ b/site/content/en/contributions/design/proxy-metrics.md @@ -1,7 +1,13 @@ --- -title: "Observability: Metrics" +title: "Data Plane Observability: Metrics" --- +This document aims to cover all aspects of envoy gateway data plane metrics observability. + +{{% alert title="Note" color="secondary" %}} +**Control plane** observability (while important) is outside of scope for this document. For control plane observability, refer to [here](./eg-metrics). +{{% /alert %}} + ## Overview Envoy provide robust platform for metrics, Envoy support three different kinds of stats: counter, gauges, histograms. diff --git a/site/content/en/contributions/design/tracing.md b/site/content/en/contributions/design/proxy-tracing.md similarity index 99% rename from site/content/en/contributions/design/tracing.md rename to site/content/en/contributions/design/proxy-tracing.md index a2790690fa6..d3542f3b74d 100644 --- a/site/content/en/contributions/design/tracing.md +++ b/site/content/en/contributions/design/proxy-tracing.md @@ -1,5 +1,5 @@ --- -title: "Observability: Tracing" +title: "Data Plane Observability: Tracing" --- ## Overview diff --git a/site/content/en/latest/tasks/observability/gateway-observability.md b/site/content/en/latest/tasks/observability/gateway-observability.md new file mode 100644 index 00000000000..6409dcee599 --- /dev/null +++ b/site/content/en/latest/tasks/observability/gateway-observability.md @@ -0,0 +1,182 @@ +--- +title: "Gateway Observability" +--- + +Envoy Gateway provides observability for the ControlPlane and the underlying EnvoyProxy instances. +This task show you how to config gateway control-plane observability, includes metrics. + +## Prerequisites + +Follow the steps from the [Quickstart](../quickstart) to install Envoy Gateway and the example manifest. +Before proceeding, you should be able to query the example backend using HTTP. + +[OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) offers a vendor-agnostic implementation of how to receive, process and export telemetry data. +Install OTel-Collector: + +```shell +helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts +helm repo update +helm upgrade --install otel-collector open-telemetry/opentelemetry-collector -f https://raw.githubusercontent.com/envoyproxy/gateway/latest/examples/otel-collector/helm-values.yaml -n monitoring --create-namespace --version 0.60.0 +``` + +## Metrics + +The default installation of Envoy Gateway installs a default [EnvoyGateway][] configuration and attaches it +using a `ConfigMap`. In this section, we will update this resource to enable various ways to retrieve metrics +from Envoy Gateway. + +### Retrieve Prometheus Metrics from Envoy Gateway + +By default, prometheus metric is enabled. You can directly retrieve metrics from Envoy Gateway: + +```shell +export ENVOY_POD_NAME=$(kubectl get pod -n envoy-gateway-system --selector=control-plane=envoy-gateway,app.kubernetes.io/instance=eg -o jsonpath='{.items[0].metadata.name}') +kubectl port-forward pod/$ENVOY_POD_NAME -n envoy-gateway-system 19001:19001 + +# check metrics +curl localhost:19001/metrics +``` + +The following is an example to disable prometheus metric for Envoy Gateway. + +{{< tabpane text=true >}} +{{% tab header="Apply from stdin" %}} + +```shell +cat <}} + +After updating the `ConfigMap`, you will need to restart the `envoy-gateway` deployment so the configuration kicks in: + +```shell +kubectl rollout restart deployment envoy-gateway -n envoy-gateway-system +``` + +### Enable Open Telemetry sink in Envoy Gateway + +The following is an example to send metric via Open Telemetry sink to OTEL gRPC Collector. + +{{< tabpane text=true >}} +{{% tab header="Apply from stdin" %}} + +```shell +cat <}} + +After updating the `ConfigMap`, you will need to restart the `envoy-gateway` deployment so the configuration kicks in: + +```shell +kubectl rollout restart deployment envoy-gateway -n envoy-gateway-system +``` + +Verify OTel-Collector metrics: + +```shell +export OTEL_POD_NAME=$(kubectl get pod -n monitoring --selector=app.kubernetes.io/name=opentelemetry-collector -o jsonpath='{.items[0].metadata.name}') +kubectl port-forward pod/$OTEL_POD_NAME -n monitoring 19001:19001 + +# check metrics +curl localhost:19001/metrics +``` + +[EnvoyGateway]: ../../api/extension_types#envoygateway diff --git a/site/content/en/latest/tasks/observability/proxy-observability.md b/site/content/en/latest/tasks/observability/proxy-observability.md index 5a1800ff39b..90cab919325 100644 --- a/site/content/en/latest/tasks/observability/proxy-observability.md +++ b/site/content/en/latest/tasks/observability/proxy-observability.md @@ -59,7 +59,7 @@ export ENVOY_POD_NAME=$(kubectl get pod -n envoy-gateway-system --selector=gatew kubectl port-forward pod/$ENVOY_POD_NAME -n envoy-gateway-system 19001:19001 # check metrics -curl localhost:19001/stats/prometheus | grep "default/backend/rule/0/match/0-www" +curl localhost:19001/stats/prometheus | grep "default/backend/rule/0" ``` You can disable metrics by setting the `telemetry.metrics.prometheus.disable` to `true` in the `EnvoyProxy` CRD. @@ -82,7 +82,7 @@ export OTEL_POD_NAME=$(kubectl get pod -n monitoring --selector=app.kubernetes.i kubectl port-forward pod/$OTEL_POD_NAME -n monitoring 19001:19001 # check metrics -curl localhost:19001/metrics | grep "default/backend/rule/0/match/0-www" +curl localhost:19001/metrics | grep "default/backend/rule/0" ``` ## Logs diff --git a/site/content/en/latest/tasks/security/threat-model.md b/site/content/en/latest/tasks/security/threat-model.md index fcf8643e184..cd5927e8d98 100644 --- a/site/content/en/latest/tasks/security/threat-model.md +++ b/site/content/en/latest/tasks/security/threat-model.md @@ -528,7 +528,7 @@ When considering internal threat actors, we chose to follow the [security model] **Threat**: Threat actors establish persistence and move laterally through the cluster unnoticed. - **Recommendation**: Configure [access logging](../../../contributions/design/accesslog) in the EnvoyProxy. Use [ProxyAccessLogFormatType](../../api/extension_types#proxyaccesslogformattype) (Text or JSON) to specify the log format and ensure that the logs are sent to the desired sink types by setting the [ProxyAccessLogSinkType](https://gateway.envoyproxy.io/latest/api/extension_types/#proxyaccesslogsinktype). Make use of [FileEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#fileenvoyproxyaccesslog) or [OpenTelemetryEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#opentelemetryenvoyproxyaccesslog) to configure File and OpenTelemetry sinks, respectively. If the settings aren\'t defined, the default format is sent to stdout. + **Recommendation**: Configure [access logging](../../../contributions/design/proxy-accesslog) in the EnvoyProxy. Use [ProxyAccessLogFormatType](../../api/extension_types#proxyaccesslogformattype) (Text or JSON) to specify the log format and ensure that the logs are sent to the desired sink types by setting the [ProxyAccessLogSinkType](https://gateway.envoyproxy.io/latest/api/extension_types/#proxyaccesslogsinktype). Make use of [FileEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#fileenvoyproxyaccesslog) or [OpenTelemetryEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#opentelemetryenvoyproxyaccesslog) to configure File and OpenTelemetry sinks, respectively. If the settings aren\'t defined, the default format is sent to stdout. Additionally, consider leveraging a central logging mechanism such as [Fluentd](https://github.com/fluent/fluentd) to enhance visibility into access activity and enable effective incident response (IR). @@ -612,7 +612,7 @@ Set runAsUser and runAsGroup security context options to specific UIDs (e.g., ru |EGTM-011|EGTM-GW-003|Gateway API| There is a risk that a gateway owner (or someone with the ability to set namespace labels) maliciously or accidentally binds routes across namespace boundaries, potentially compromising the confidentiality and integrity of traffic in a multitenant scenario.

| If a Route Binding within a Gateway Listener is configured based on a custom label, it could allow a malicious internal actor with the ability to label namespaces to change the set of namespaces supported by the Gateway

|Low| Consider the use of custom admission control to restrict what labels can be set on namespaces through tooling such as [Kubewarden](https://kyverno.io/policies/pod-security/), [Kyverno](https://github.com/kubewarden), and [OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper). Route binding should follow the Kubernetes Gateway API security model, as shown [here](https://gateway-api.sigs.k8s.io/concepts/security-model/#1-route-binding), to connect gateways in different namespaces. | |EGTM-013|EGTM-GW-005|Gateway API| There is a risk that an unauthorised actor deploys an unauthorised GatewayClass due to GatewayClass namespace validation not being configured, leading to non-compliance with business and security requirements.

| Unauthorised deployment of Gateway resource via GatewayClass template which crosses namespace trust boundaries.

|Low| Leverage GatewayClass namespace validation to limit the namespaces where GatewayClasses can be run through a tool such as using [OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper). Reference pull request \#[24](https://github.com/open-policy-agent/gatekeeper-library/pull/24) within gatekeeper-library which outlines how to add GatewayClass namespace validation through a GatewayClassNamespaces API resource kind within the constraints.gatekeeper.sh/v1beta1 apiGroup. | |EGTM-015|EGTM-CS-007|Container Security| There is a risk that threat actors could exploit ServiceAccount tokens for illegitimate authentication, thereby leading to privilege escalation and the undermining of gateway API resources\' integrity, confidentiality, and availability.

| The threat arises from threat actors impersonating the envoy-gateway ServiceAccount through the replay of ServiceAccount tokens, thereby achieving escalated privileges and gaining unauthorised access to Kubernetes resources.

|Low| Limit the creation of ServiceAccounts to only when necessary, specifically refraining from using default service account tokens, especially for high-privilege service accounts. For legacy clusters running Kubernetes version 1.21 or earlier, note that ServiceAccount tokens are long-lived by default. To disable the automatic mounting of the service account token, set automountServiceAccountToken: false in the PodSpec. | -|EGTM-016|EGTM-EG-004|Envoy Gateway| There is a risk that threat actors establish persistence and move laterally through the cluster unnoticed due to limited visibility into access and application-level activity.

| Threat actors establish persistence and move laterally through the cluster unnoticed.

|Low| Configure [access logging](../../../contributions/design/accesslog) in the EnvoyProxy. Use [ProxyAccessLogFormatType](../../api/extension_types#proxyaccesslogformattype) (Text or JSON) to specify the log format and ensure that the logs are sent to the desired sink types by setting the [ProxyAccessLogSinkType](https://gateway.envoyproxy.io/latest/api/extension_types/#proxyaccesslogsinktype). Make use of [FileEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#fileenvoyproxyaccesslog) or [OpenTelemetryEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#opentelemetryenvoyproxyaccesslog) to configure File and OpenTelemetry sinks, respectively. If the settings aren\'t defined, the default format is sent to stdout.

Additionally, consider leveraging a central logging mechanism such as [Fluentd](https://github.com/fluent/fluentd) to enhance visibility into access activity and enable effective incident response (IR). | +|EGTM-016|EGTM-EG-004|Envoy Gateway| There is a risk that threat actors establish persistence and move laterally through the cluster unnoticed due to limited visibility into access and application-level activity.

| Threat actors establish persistence and move laterally through the cluster unnoticed.

|Low| Configure [access logging](../../../contributions/design/proxy-accesslog) in the EnvoyProxy. Use [ProxyAccessLogFormatType](../../api/extension_types#proxyaccesslogformattype) (Text or JSON) to specify the log format and ensure that the logs are sent to the desired sink types by setting the [ProxyAccessLogSinkType](https://gateway.envoyproxy.io/latest/api/extension_types/#proxyaccesslogsinktype). Make use of [FileEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#fileenvoyproxyaccesslog) or [OpenTelemetryEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#opentelemetryenvoyproxyaccesslog) to configure File and OpenTelemetry sinks, respectively. If the settings aren\'t defined, the default format is sent to stdout.

Additionally, consider leveraging a central logging mechanism such as [Fluentd](https://github.com/fluent/fluentd) to enhance visibility into access activity and enable effective incident response (IR). | |EGTM-017|EGTM-EG-005|Envoy Gateway| There is a risk that an insider misconfigures an envoy gateway component and goes unnoticed due to a low-touch logging configuration (via default) which responsible stakeholders are not aptly aware of or have immediate access to.

| The threat emerges from an insider misconfiguring an Envoy Gateway component without detection.

|Low| Configure the logging level of the Envoy Gateway using the \'level\' field in [EnvoyGatewayLogging](https://gateway.envoyproxy.io/latest/api/extension_types/#envoygatewaylogging). Ensure the appropriate logging levels are set for relevant components such as \'gateway-api\', \'xds-translator\', or \'global-ratelimit\'. If left unspecified, the logging level defaults to \"info\", which may not provide sufficient detail for security monitoring.

Employ a centralised logging mechanism, like [Fluentd](https://github.com/fluent/fluentd), to enhance visibility into application-level activity and to enable efficient incident response. | |EGTM-021|EGTM-EG-006|Envoy Gateway| There is a risk that the admin interface is exposed without valid business reason, increasing the attack surface.

| Exposed admin interfaces give internal attackers the option to affect production traffic in unauthorised ways, and the option to exploit any vulnerabilities which may be present in the admin interface (e.g. by orchestrating malicious GET requests to the admin interface through CSRF, compromising Envoy Proxy global configuration or shutting off the service entirely (e.g., /quitquitquit).

|Low| The Envoy Proxy admin interface is only exposed to localhost, meaning that it is secure by default. However, due to the risk of misconfiguration, this recommendation is included.

Due to the importance of the admin interface, it is recommended to ensure that Envoy Proxies have not been accidentally misconfigured to expose the admin interface to untrusted networks. | |EGTM-025 | EGTM-CS-011 | Container Security | The presence of a vulnerability, be it in the kernel or another system component, when coupled with containers running as root, could enable a threat actor to escape the container, thereby compromising the confidentiality, integrity, or availability of cluster resources. | The Envoy Proxy container's root-user configuration can be leveraged by an attacker to escalate privileges, execute a container breakout, and traverse across trust boundaries. | Low | By default, Envoy Gateway deployments do not use root users. Nonetheless, in case a custom image or deployment manifest is to be used, make sure Envoy Proxy pods run as a non-root user with a high UID within the container. Set runAsUser and runAsGroup security context options to specific UIDs (e.g., runAsUser: 1000 & runAsGroup: 3000) to ensure the container operates with the stipulated non-root user and group ID. If using helm chart deployment, define the user and group ID in the values.yaml file or via the command line during helm install / upgrade. | diff --git a/site/content/en/v1.0.1/tasks/security/threat-model.md b/site/content/en/v1.0.1/tasks/security/threat-model.md index 79a69b482c6..c1bba3f9726 100644 --- a/site/content/en/v1.0.1/tasks/security/threat-model.md +++ b/site/content/en/v1.0.1/tasks/security/threat-model.md @@ -528,7 +528,7 @@ When considering internal threat actors, we chose to follow the [security model] **Threat**: Threat actors establish persistence and move laterally through the cluster unnoticed. - **Recommendation**: Configure [access logging](../../../contributions/design/accesslog) in the EnvoyProxy. Use [ProxyAccessLogFormatType](../../api/extension_types#proxyaccesslogformattype) (Text or JSON) to specify the log format and ensure that the logs are sent to the desired sink types by setting the [ProxyAccessLogSinkType](https://gateway.envoyproxy.io/latest/api/extension_types/#proxyaccesslogsinktype). Make use of [FileEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#fileenvoyproxyaccesslog) or [OpenTelemetryEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#opentelemetryenvoyproxyaccesslog) to configure File and OpenTelemetry sinks, respectively. If the settings aren\'t defined, the default format is sent to stdout. + **Recommendation**: Configure [access logging](../../../contributions/design/proxy-accesslog) in the EnvoyProxy. Use [ProxyAccessLogFormatType](../../api/extension_types#proxyaccesslogformattype) (Text or JSON) to specify the log format and ensure that the logs are sent to the desired sink types by setting the [ProxyAccessLogSinkType](https://gateway.envoyproxy.io/latest/api/extension_types/#proxyaccesslogsinktype). Make use of [FileEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#fileenvoyproxyaccesslog) or [OpenTelemetryEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#opentelemetryenvoyproxyaccesslog) to configure File and OpenTelemetry sinks, respectively. If the settings aren\'t defined, the default format is sent to stdout. Additionally, consider leveraging a central logging mechanism such as [Fluentd](https://github.com/fluent/fluentd) to enhance visibility into access activity and enable effective incident response (IR). @@ -612,7 +612,7 @@ Set runAsUser and runAsGroup security context options to specific UIDs (e.g., ru |EGTM-011|EGTM-GW-003|Gateway API| There is a risk that a gateway owner (or someone with the ability to set namespace labels) maliciously or accidentally binds routes across namespace boundaries, potentially compromising the confidentiality and integrity of traffic in a multitenant scenario.

| If a Route Binding within a Gateway Listener is configured based on a custom label, it could allow a malicious internal actor with the ability to label namespaces to change the set of namespaces supported by the Gateway

|Low| Consider the use of custom admission control to restrict what labels can be set on namespaces through tooling such as [Kubewarden](https://kyverno.io/policies/pod-security/), [Kyverno](https://github.com/kubewarden), and [OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper). Route binding should follow the Kubernetes Gateway API security model, as shown [here](https://gateway-api.sigs.k8s.io/concepts/security-model/#1-route-binding), to connect gateways in different namespaces. | |EGTM-013|EGTM-GW-005|Gateway API| There is a risk that an unauthorised actor deploys an unauthorised GatewayClass due to GatewayClass namespace validation not being configured, leading to non-compliance with business and security requirements.

| Unauthorised deployment of Gateway resource via GatewayClass template which crosses namespace trust boundaries.

|Low| Leverage GatewayClass namespace validation to limit the namespaces where GatewayClasses can be run through a tool such as using [OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper). Reference pull request \#[24](https://github.com/open-policy-agent/gatekeeper-library/pull/24) within gatekeeper-library which outlines how to add GatewayClass namespace validation through a GatewayClassNamespaces API resource kind within the constraints.gatekeeper.sh/v1beta1 apiGroup. | |EGTM-015|EGTM-CS-007|Container Security| There is a risk that threat actors could exploit ServiceAccount tokens for illegitimate authentication, thereby leading to privilege escalation and the undermining of gateway API resources\' integrity, confidentiality, and availability.

| The threat arises from threat actors impersonating the envoy-gateway ServiceAccount through the replay of ServiceAccount tokens, thereby achieving escalated privileges and gaining unauthorised access to Kubernetes resources.

|Low| Limit the creation of ServiceAccounts to only when necessary, specifically refraining from using default service account tokens, especially for high-privilege service accounts. For legacy clusters running Kubernetes version 1.21 or earlier, note that ServiceAccount tokens are long-lived by default. To disable the automatic mounting of the service account token, set automountServiceAccountToken: false in the PodSpec. | -|EGTM-016|EGTM-EG-004|Envoy Gateway| There is a risk that threat actors establish persistence and move laterally through the cluster unnoticed due to limited visibility into access and application-level activity.

| Threat actors establish persistence and move laterally through the cluster unnoticed.

|Low| Configure [access logging](../../../contributions/design/accesslog) in the EnvoyProxy. Use [ProxyAccessLogFormatType](../../api/extension_types#proxyaccesslogformattype) (Text or JSON) to specify the log format and ensure that the logs are sent to the desired sink types by setting the [ProxyAccessLogSinkType](https://gateway.envoyproxy.io/latest/api/extension_types/#proxyaccesslogsinktype). Make use of [FileEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#fileenvoyproxyaccesslog) or [OpenTelemetryEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#opentelemetryenvoyproxyaccesslog) to configure File and OpenTelemetry sinks, respectively. If the settings aren\'t defined, the default format is sent to stdout.

Additionally, consider leveraging a central logging mechanism such as [Fluentd](https://github.com/fluent/fluentd) to enhance visibility into access activity and enable effective incident response (IR). | +|EGTM-016|EGTM-EG-004|Envoy Gateway| There is a risk that threat actors establish persistence and move laterally through the cluster unnoticed due to limited visibility into access and application-level activity.

| Threat actors establish persistence and move laterally through the cluster unnoticed.

|Low| Configure [access logging](../../../contributions/design/proxy-accesslog) in the EnvoyProxy. Use [ProxyAccessLogFormatType](../../api/extension_types#proxyaccesslogformattype) (Text or JSON) to specify the log format and ensure that the logs are sent to the desired sink types by setting the [ProxyAccessLogSinkType](https://gateway.envoyproxy.io/latest/api/extension_types/#proxyaccesslogsinktype). Make use of [FileEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#fileenvoyproxyaccesslog) or [OpenTelemetryEnvoyProxyAccessLog](https://gateway.envoyproxy.io/latest/api/extension_types/#opentelemetryenvoyproxyaccesslog) to configure File and OpenTelemetry sinks, respectively. If the settings aren\'t defined, the default format is sent to stdout.

Additionally, consider leveraging a central logging mechanism such as [Fluentd](https://github.com/fluent/fluentd) to enhance visibility into access activity and enable effective incident response (IR). | |EGTM-017|EGTM-EG-005|Envoy Gateway| There is a risk that an insider misconfigures an envoy gateway component and goes unnoticed due to a low-touch logging configuration (via default) which responsible stakeholders are not aptly aware of or have immediate access to.

| The threat emerges from an insider misconfiguring an Envoy Gateway component without detection.

|Low| Configure the logging level of the Envoy Gateway using the \'level\' field in [EnvoyGatewayLogging](https://gateway.envoyproxy.io/latest/api/extension_types/#envoygatewaylogging). Ensure the appropriate logging levels are set for relevant components such as \'gateway-api\', \'xds-translator\', or \'global-ratelimit\'. If left unspecified, the logging level defaults to \"info\", which may not provide sufficient detail for security monitoring.

Employ a centralised logging mechanism, like [Fluentd](https://github.com/fluent/fluentd), to enhance visibility into application-level activity and to enable efficient incident response. | |EGTM-021|EGTM-EG-006|Envoy Gateway| There is a risk that the admin interface is exposed without valid business reason, increasing the attack surface.

| Exposed admin interfaces give internal attackers the option to affect production traffic in unauthorised ways, and the option to exploit any vulnerabilities which may be present in the admin interface (e.g. by orchestrating malicious GET requests to the admin interface through CSRF, compromising Envoy Proxy global configuration or shutting off the service entirely (e.g., /quitquitquit).

|Low| The Envoy Proxy admin interface is only exposed to localhost, meaning that it is secure by default. However, due to the risk of misconfiguration, this recommendation is included.

Due to the importance of the admin interface, it is recommended to ensure that Envoy Proxies have not been accidentally misconfigured to expose the admin interface to untrusted networks. | |EGTM-025 | EGTM-CS-011 | Container Security | The presence of a vulnerability, be it in the kernel or another system component, when coupled with containers running as root, could enable a threat actor to escape the container, thereby compromising the confidentiality, integrity, or availability of cluster resources. | The Envoy Proxy container's root-user configuration can be leveraged by an attacker to escalate privileges, execute a container breakout, and traverse across trust boundaries. | Low | By default, Envoy Gateway deployments do not use root users. Nonetheless, in case a custom image or deployment manifest is to be used, make sure Envoy Proxy pods run as a non-root user with a high UID within the container. Set runAsUser and runAsGroup security context options to specific UIDs (e.g., runAsUser: 1000 & runAsGroup: 3000) to ensure the container operates with the stipulated non-root user and group ID. If using helm chart deployment, define the user and group ID in the values.yaml file or via the command line during helm install / upgrade. |