[APIM] Enable the apim-metrics.log without enabling the ELK analytics type

[Ramiiyan]

Problem

In the current versions of WSO2 API Manager (4.x.x), users can view API invocation-related data in a single line within the apim-metrics.log file when ELK analytics is enabled. This log includes all relevant API analytics data, such as response codes, client-IPs, and more.

Eg:
15:56:07,928 [-] [PassThroughMessageProcessor-18] INFO ELKCounterMetric apimMetrics: apim:response, properties :{"apiName":"PizzaShackAPI","proxyResponseCode":200,"destination":"https://localhost:9443/am/sample/pizzashack/v1/api/","apiCreatorTenantDomain":"carbon.super","platform":"Mac OS X","apiMethod":"GET","apiVersion":"1.0.0","gatewayType":"SYNAPSE","apiCreator":"admin","responseCacheHit":false,"backendLatency":33,"correlationId":"2f8daa81-b7cf-49d2-be4b-f48be7b2b4e6","requestMediationLatency":5,"keyType":"PRODUCTION","apiId":"cbd025f7-d3db-4513-9f0b-1d9a8dc1f54f","applicationName":"DefaultApplication","targetResponseCode":200,"requestTimestamp":"2024-03-19T10:26:07.888Z","applicationOwner":"admin","userAgent":"Chrome","userName":"admin@carbon.super","apiResourceTemplate":"/menu","responseLatency":39,"regionId":"default","responseMediationLatency":1,"userIp":"127.0.0.1","apiContext":"/pizzashack/1.0.0","applicationId":"0ade0205-1260-4446-9b75-8352569d1f9e","apiType":"HTTP","properties":{}}

However, this feature is only available when using ELK analytics.

When Choreo analytics is configured as the primary analytics engine for APIM, users cannot enable the apim-metrics.log feature. This creates a gap in monitoring, as Choreo analytics does not provide the same logging information as source-ip in the apim-metrics.log file, making it difficult.

Solution

Enable the apim-metrics.log feature separately for APIM users even when Choreo analytics or ELK is configured or not. This would involve:

1. Adding a Configuration Option: Allow APIM to log API invocation-related data to apim-metrics.log independently of the chosen analytics engine (whether ELK or Choreo).
2. Uniform Logging Format: Ensure that the log format remains consistent with the format used when ELK analytics is enabled, providing a single-line summary of API invocation data.

Affected Component

Analytics

Version

4.x.x

Implementation

No response

Related Issues

No response

Suggested Labels