Ideal value for GPU_MEMORY_UTILIZATION

[houmie]

What is the ideal value for GPU_MEMORY_UTILIZATION when memory is a bit tight? Should I increase it to 0.99 to give the LLM model more room to expand, or will stability improve if the value remains at 0.95?

Is my assumption correct that increasing the memory allocation to 0.99 would enhance performance, but it also poses a slight risk of an out-of-memory error? This is paradoxical, as if the model doesn't fit into RAM, it is expected to throw an out-of-memory error.