How to re-enqueue job or restart worker gracefully when the processing worker was crashed or shutdown

[zshnb]

Is your feature request related to a problem? Please describe.
I have some workers which is running in google cloud spot, the spot will be recycled by google cloud anytime,
therefore if a worker is processing a job, like read/write files, download/upload files, then the job will be shutdown or
throw some error, for example, if the worker will execute uploadToGcs next, but didn't execute. so i want to ask how to deal with
above case, is there any way to restart worker in anytime gracefully

Describe the solution you'd like
restart worker in anytime gracefully or re-enqueue job

Describe alternatives you've considered
none

Additional context
none

[manast]

In BullMQ, jobs that are active when a worker is unexpectedly shut down will become stalled and processed again by another worker. Maybe you can increase the number of stalled attempts if this is something expected to happen often. I guess you can also implement inside your worker some process shutdown detection and try to stop doing what you are doing in a more graceful way, I am not sure how much time you get from the instant you get a SIGTERM from GCP until the instance actually gets shutdown... anyway, I think there is not so much BullMQ can do, you need to handle this the best way possible either with shutdown detection inside the workers or just relying on the stalled retries. https://api.docs.bullmq.io/interfaces/WorkerOptions.html#maxStalledCount

[zshnb]

yeah, increase stall count is a good idea, i'll try it. for another solution, i'm not sure if GCP's shutdown spot will wait my custom process signal handler finish