Top client user report

[razaahmed]

Is it possible to get statistics for top client users report in harvest?

Here is the KB for this data that manually generated on demand when need to trobleshoot slowness or whatever issue lead to find who is the top talker:

https://kb.netapp.com/onprem/ontap/da/NAS/How_to_get_statistics_for_top_client_users_in_ONTAP_9
>statistics start -object top_client -sort-key write_ops -sample-id writeop1
>statistics show -object top_client -sample-id writeop1 -tab -counter read_ops|write_ops -sort-key write_ops
>>statistics stop -sample-id writeop1

[cgrinds]

Thanks @razaahmed for opening this issue that started on Discord.

[Sandromuc]

We would like to have this too
+1

[mamoep]

+1

[wooyoungAhn]

We would like to have this too
+1

[razaahmed]

Any update on this?

[rahulguptajss]

@razaahmed Latest update on this feature is as below.

Let's take the top client as an example, which can be collected via the endpoints /storage/volumes/*/top-metrics for volume level or /svm/svms/{svm.uuid}/top-metrics at the SVM level. At the volume level, it will be an expensive call for ONTAP, especially when using the wildcard *. For SVMs, we will also need to make multiple calls to ONTAP to collect these stats. Also, each metric, such as iops.read, iops.write, throughput.read, and throughput.write, must be queried separately, which would quadruple the number of API calls.

Even if we make these calls, given the nature of the unique time series, this may generate for Prometheus, it is not good for Prometheus cardinality (https://last9.io/blog/how-to-manage-high-cardinality-metrics-in-prometheus/). Every time we may get different top clients, resulting in a lot of time series generation in Prometheus.

One of the possible solution for this is to not publish these metrics to Prometheus but to expose them via the Harvest CLI. We can call SVM endpoints via the CLI and display results via stdout only. But that is not something we are planning to implement currently.

[cgrinds]

In short, it's an ONTAP API limitation that makes it a challenge to collect these metrics.

[rahulguptajss]

@razaahmed We have found a solution to this issue. Could you please let us know your ONTAP version? We intend to make it compatible with the RestPerf collector.

[rahulguptajss]

@razaahmed @mamoep @wooyoungAhn @Sandromuc
This feature is now available through nightly build. For more details, please refer to the discussion here.