Windows agent gets unhealthy on adding Elastic Defend integration.

[amolnater-qasource]

Kibana Build details:

VERSION: 8.16.0 SNAPSHOT
BUILD: 78344
COMMIT: ec719e0c2adbf707701892198743dd7b263a5b67

Artifact: https://snapshots.elastic.co/8.16.0-8f34d333/downloads/beats/elastic-agent/elastic-agent-8.16.0-SNAPSHOT-windows-x86_64.zip

Host: Windows Server 2022- Test Signing ON

Preconditions:

1. 8.16.0 SNAPSHOT Kibana cloud environment should be available.
2. Agent should be installed with policy having System and Elastic Defend integrations.

Steps to reproduce:

1. Navigate to Agents tab.
2. Observe the Agent is unhealthy.
3. Navigate to Agent>Logs tab.
4. Set logging level filter to error
5. Observe errors related ElasticEndpoint.

Expected Result:
Windows agent should remain healthy on adding Elastic Defend integration.

Logs:
elastic-agent-diagnostics-2024-09-18T08-20-53Z-00.zip

Screenshots:

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[amolnater-qasource]

@muskangulati-qasource Please review.

[muskangulati-qasource]

Secondary review is Done for this ticket!!

[cmacknz]

The only transition to degraded I see is:

logs/elastic-agent-8.16.0-SNAPSHOT-8edddc/elastic-agent-20240918-1.ndjson
405:{"log.level":"warn","@timestamp":"2024-09-18T08:13:05.839Z","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/agent/application/coordinator.(*Coordinator).watchRuntimeComponents","file.name":"coordinator/coordinator.go","file.line":663},"message":"Unit state changed system/metrics-monitoring-metrics-monitoring-endpoint_security (HEALTHY->DEGRADED): Error fetching data for metricset system.process: error fetching pid 4884: Not enough privileges to fetch information: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.","log":{"source":"elastic-agent"},"component":{"id":"system/metrics-monitoring","state":"HEALTHY"},"unit":{"id":"system/metrics-monitoring-metrics-monitoring-endpoint_security","type":"input","state":"DEGRADED","old_state":"HEALTHY"},"ecs.version":"1.6.0"}

        units:
            input-system/metrics-monitoring-metrics-monitoring-endpoint_security:
                message: 'Error fetching data for metricset system.process: error fetching pid 4884: Not enough privileges to fetch information: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.'
                payload:
                    streams:
                        metrics-monitoring-endpoint_security:
                            error: 'Error fetching data for metricset system.process: error fetching pid 4884: Not enough privileges to fetch information: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.'
                            status: DEGRADED
                state: 3
            output-system/metrics-monitoring:
                message: Healthy
                state: 2
                        system/metrics-system.process-26525289-26d7-4040-9000-bd032324d2a3:
                            error: |-
                                Error fetching data for metricset system.process: Not enough privileges to fetch information: Not enough privileges to fetch information: GetInfoForPid: could not get all information for PID 0: error fetching name: OpenProcess failed for pid=0: The parameter is incorrect.
                                error fetching status: OpenProcess failed for pid=0: The parameter is incorrect.
                                GetInfoForPid: could not get all information for PID 4: error fetching name: GetProcessImageFileName failed for pid=4: GetProcessImageFileName failed: invalid argument
                                non fatal error fetching PID some info for 100, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 444, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 600, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 672, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 680, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 816, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 2500, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 4908, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 3560, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                                non fatal error fetching PID some info for 4884, metrics are valid, but partial: FillMetricsRequiringMoreAccess: error fetching process args: Not enough privileges to fetch information: OpenProcess failed: Access is denied.
                            status: HEALTHY

[VihasMakwana]

Relates: elastic/beats#40484

[VihasMakwana]

elastic/beats#40924 should fix this. I'll test it from my side on my windows machine and keep you posted.

cc: @pierrehilbert @cmacknz @ycombinator

[VihasMakwana]

@amolnater-qasource the fix has been merged.

[amolnater-qasource]

Hi @VihasMakwana

We have revalidated this issue on latest 8.16.0 SNAPSHOT and found it fixed now.

Observations:

• Windows agent remains healthy on adding Elastic Defend integration.

Build details:

VERSION: 8.16.0 SNAPSHOT
BUILD: 78938
COMMIT: 7b832691e8b07c67b411da95b0398a04711da864

Artifact: https://snapshots.elastic.co/8.16.0-39df64b4/downloads/beats/elastic-agent/elastic-agent-8.16.0-SNAPSHOT-windows-x86_64.zip

Screenshots:

Logs:
elastic-agent-diagnostics-2024-10-09T08-56-15Z-00.zip

Hence, we are closing & marking this issue as QA:Validated.

Thanks!!