Optimize user count retrieval logic in user filter operation to reduce extensive DB calls

[bhagyasakalanka]

Is your suggestion related to an experience ? Please describe.
During user listing, we observed inefficiencies with numerous extensive database calls. For instance, to retrieve the total user count, all user records are fetched instead of just the count. Additionally, there are multiple duplicate calls contributing to the overhead.

Describe the improvement
A new method to retrieve the total user count was introduced to the kernel with PR [1]. However, due to the added complexities in the SCIM2 component, this method cannot be directly applied at the SCIM layer.

To address this, we have raised PR [2] to extend the kernel’s capability to the SCIM layer. However, because of the complexities involved, directly integrating this method would disrupt existing behavior. As a result, this improvement needs to be considered carefully to ensure smooth implementation without breaking existing functionality at the SCIM level.

[1] wso2/carbon-kernel#4033
[2] wso2-extensions/identity-inbound-provisioning-scim2#562

[bhagyasakalanka]

This is complete with: wso2-extensions/identity-inbound-provisioning-scim2#579