does plotman account for ram before starting?

[syadnom]

I'm wondering if plotman accounts for RAM before starting a plot and/or suspending. I just spun up a small machine with 16GB of RAM but enough IO to do 6 plots at a time. I have it configured for 4 plots so I don't overcommit RAM. I'd like to do 6 if plotman will handle it. There are phases that use much less RAM and I don't want to gamble on running out of RAM.

thoughts?

[altendky]

No, plotman does not have a RAM limiter. You think there are portions of plotting that use significantly under 4GiB of RAM? I can't say I've looked into this a whole lot.

Some random person's resource usage log...
https://i.redd.it/w0jl1k9ml8t61.png

[syadnom]

Yes, 1:1 for instance uses <500M, 1:2 allocates 4.1G but doesn't actually use all of it, with 4 plots it's using 10.4G on this machine through the rest of phase 1. phase 2 allocates 1.4-1.6GB while phase 3 allocates 4.5. Actual usage seems to be about 10-20% lower than it allocates.

If everything was timed perfectly, I think you could get 6 plots out of 16GB though there would be delays because you'd have to pause jobs before they hit the heavy sort in phase 3. That seems to be the highest actual usage, phase 3.1. 3:5 allocates 3.9G.

I have an r710 with 23 drives and I have 20 plots running on it. That should demand about 68GB of ram but because the drives are different speeds etc this machine has plots spread out on various phases and my peak ram usage is about 38G as reported in htop and free.

I have an optiplex with 16G and 4 plots, all in phase 1, subphases 3,4,4, and 4. RAM at 8.51G

What would be interesting is if at the end of each phase, plotman would check and see if there is enough ram to start the next phase or not and pause the job if not. Same when starting a job. Also add up all phases for a job and make sure you don't stage it into a corner. Ie, cant have too many plots hitting phase 3.1 where RAM use is the highest or too many will sit paused at the end of phase 2.

[syadnom]

I'm going to try just having enough swap to overcome short term memory overcommit and also take to top 1-3 plotters and do a cgroup/swappiness=0 to try to keep them in ram and let processes further back swap a bit.

MOST of the time I'm using well under the perscribed ram but I suppose there is a change that everything aligns during phase 3:1 when RAM demand is the highest.

to expand on what I said before. I have 5 plots running on this machine now and RAM is sitting at 6.55GB. phases are 1:1,1:6,2:3,2:3,2:3. I may back this one down to 4 plots depending on CPU. This gen3 intel quad core is getting fully utilized so memory pressure might not be my bottleneck here.