Implement some overprovisioning protections at a storage-pool level

[dvance]

The MaxOversubscriptionRatio works on a per-volume basis. This means that, with for example a storage-pool 1TiB in size and a ratio of 2x, Linstor would only allow us to create a volume <2TiB in size. However, you could still create 20 resources all of which are 1TiB in size.

Would it be possible for there to be some feature that prevents making an additional resource if the total aggregate size of all resource using that storage pool was to total more than 2x the storage pool size (2TiB in this example).

[ghernadi]

TL;DR: It's complicated 🙂

Slightly longer overview of how it currently works and hightlighting some issues:
Sure, having a 1TiB storage, setting an oversubscription ratio of 2.0 to allow 2TiB of volume reservation sounds easy
enough. In theory at least. In practice things are a bit different, but let's investigate this scenario a bit further.

The previous statement is easy enough for an empty storage pool. However, things are getting complicated when considering
partially (or almost completely) filled storage pools. For example, if we stick with out 1TiB storage pool, but assume
that 500GiB are already reserved. Should LINSTOR still allow additional 1.5TiB of volumes? Sure, as long as we are only
talking about thin reservations, why not, that is actually the point of overprovisioning, right?

The problem starts when we start to also consider actually used space, besides the reservations. With a 1TiB storage pool,
if we already have for example 1.5TiB reserved, but actually 950GiB allocated (i.e. in use), one can quite easily argue
that it would be a mistake if LINSTOR would allow another 500GiB volume, since LINSTOR knows that there are only 50GiB
free, right?

That's the reason why LINSTOR takes the remaining free space of the storage pool (so in the above example the 50GiB)
and applies the oversubscription ratio to only this free space, resulting in allowing another 100GiB volume instead of
a 500GiB volume.

This might be better for the above described example, but I do agree that it does not really help with the initial scenario
you described. Exactly because LINSTOR is now considering the remaining free space LINSOTR allows the 20 (and more)
1TiB resources as long as those resources do not allocate actual data.

I guess we could improve LINSTOR by implementing both approaches, the free space based as well as the reserved space
based and simply take the lower of those two values as the capacity.
That means that even with empty resources, LINSTOR would only allow up to 2 TiB of reservation in your example (i.e.
only 2x 1TiB resources), and still behave similarly in the example I gave with the 1.5TiB reserved with 950GiB already
allocated, allowing only 100GiB for a new resource.

Discussions / suggestions are welcome 🙂