[umf] optimize bucket selection for allocation sizes

[DamianDuy]

This PR changes linear search for buckets to calculating the bucket index based on calculating the power of two (or halfway power of two) larger than Size parameter by using bitwise operations.

Solves: #871

[pbalcer]

This affects SYCL, so please follow the process outlined in the contributing guide.

[pbalcer]

hm, can you please do a micro-benchmark to see which implementation is better? My initial thinking was something like 0f296e7 - that's the common approach to this problem.

[igchor]

Is CutOff exposed somehow to the user? If not, how will user know what is the max size he can set?
If this is not exposed, I think you should just set MinBucketSize to CutOff if it's bigger.

[DamianDuy]

Done.

[igchor]

hm, can you please do a micro-benchmark to see which implementation is better? My initial thinking was something like 0f296e7 - that's the common approach to this problem.

You could also try moving the size outside of the bucket class (create a second array/vector of the same size as the current Buckets that would only keep size) and do a binary search on that. This way you would have much fewer cache misses.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5c0557e) 16.19% compared to head (7602d17) 16.26%.
Report is 4 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #994      +/-   ##
==========================================
+ Coverage   16.19%   16.26%   +0.07%     
==========================================
  Files         220      221       +1     
  Lines       30792    30818      +26     
  Branches     3481     3487       +6     
==========================================
+ Hits         4986     5012      +26     
  Misses      25755    25755              
  Partials       51       51              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[DamianDuy]

After microbenchmarking I changed the implementation from binary search (which was much slower) to the approach with calculating index using bitwise operations.

[igchor]

The compiler might already do this but you could avoid the branches entirely by implementing it like this:

auto isPowerOf2 = 0 == (Size & (Size - 1));
auto largerThanHalfwayBetweenPowersOf2 = !isPowerOf2 && bool((Size - 1) && (1 << (position - 1)));
auto index = (position - MinBucketSizeExp) * 2 + !isPowerOf2 + largerThanHalfwayBetweenPowersOf2;
return index;

(if you decide to use this code, you should check if it's actually correct ;d)

[DamianDuy]

Unfortunately, it doesn't work correctly with halfway values and personally I find if branches more readable. Also wouldn't this code branch anyway because of a usage of the && operator?

[DamianDuy]

I am closing this PR because there is a corresponding one in the unified-memory-framework repositorium:
oneapi-src/unified-memory-framework#37