Not submitting partials to pools

[ekersey]

Just looked at summary page and noticed partials were showing 0.00/hr. for both chia and chives. Looks like I caught it right at 24 hours since last submission. Any idea what could cause this? I saw this once before actually after about 3 days of no submissions but it was when I was still getting things configured initially and just thought it was something I must have did wrong (would have been main branch at that time). Using test branch now.

I'll share some screenshots of the summary pages and the flexpool dashboard showing 0 points in the last 24 hours to demonstrate what I'm talking about. Otherwise I'm not really sure where to even start. Nothing in the logs or alerts jumps out at me. Any ideas would be greatly appreciated. I'm fairly certain things will work again after I docker-compose down/up when current plotting jobs finish. But I'd really like to know how to prevent this random stoppage in the first place.

[guydavis]

Hi, I would concentrate on the Chia debug.log at /mnt/user/appdata/machinaris/mainnet/log/debug.log, looking for errors. It's possible the plotting resource usage is consuming too much, resulting in missed pool submissions from resource-starved farming processes.

Some things to try:

docker exec -it machinaris bash
cd /root/.chia/mainnet/log
grep -C2 -i partial debug.log.1 debug.log

Look for succesful/stale/error partial submissions. See the Alerts page of the Machinaris WebUI as well. Hope this helps.

[ekersey]

There are no errors or warnings in either the chiadog or chia log files to indicate a problem. With the grep filter you provided, I found the last partial submission was 2022-01-31T22:19:29.334.

I think I found the problem though. Something modified the main config.yaml (fortunately a backup was created when it did).
There were 3 changes that I did not make:
target_peer_count was reduced to 20 from 80
multiprocessing_limit was removed completely
and the most important one:
pool_list: was removed completely

[ekersey]

Same thing happened to the chives config.yaml

[guydavis]

Hmm, the peer count change just sounds like Forktools. See the log at /tmp/forktools.log inside container on start. Also Settings | Tools page.

Forktools should not have removed a pool section. That would be the very first I've heard of that happening.

[Bafff]

@guydavis nope, Just remember I Have the same problems too not long ago

[Bafff]

https://discord.com/channels/842422792593866782/888076333828161597/934899550043385876

[ekersey]

I wish I would have looked at the forktools log file before fixing the config.yaml and restarting the container, maybe there would have been a clue. As it stands however, after fixing config.yaml manually and restarting containers, everything seems to be back to normal. I've modified the farming config and the forktools config and restarted containers a couple of times. I've changed settings (peer count) manually and watched forktools change it back. So far, the pool_list has persisted

Strongly suspect that it was forktools that removed pool_list, though no idea why. The reason I suspect forktools, is that the backup files it creates when it modifies config.yaml are named in a format that is slightly different than the naming format Machinaris uses. The last backup file Machinaris created was over a week ago (never once in the case of chives). And the backups show that pool_list was removed last night.