Do flow children & parents retry when failParentOnFailure = true?

[nggonzalez]

Questions

After reading the docs and open issues, I'm still struggling to understand the following:

1. Do flow child jobs with failParentOnFailure = true retry before failing the parent or do they immediately fail the parent on the first failure?
2. Do flow parent jobs retry when they are failed by a child?

In addition, Is there a way to run a flow parent when there is a failure in order to capture the error state? I've referenced this issue , but struggled to follow the discussion. Based on the linked workaround, I'm wondering if something like this would work to maintain retries + call to failed parent, assuming the flow only has one set of children (no nesting):

Pseudocode:

worker.on("failed", (job) => {
  if (job.parent) {
    job.opts.failParentOnFailure = true;
    job.retry("failed")
  }
})

// ... 

parentWorker.on("failed", (job) => {
  if (job) {
   const { unprocessed } = await job.getDependencies()
    if (unprocessed.length > 0) {
      // remove failed parent
      await job.remove();
   
       // re-add failed parent job with some new data
       await parentQueue.add(job.name, { ...job.data, status: "error" })
    }
  }
})

References

The below are issues that seem to be related, but didn't have clear answers

• https://docs.bullmq.io/guide/flows/fail-parent
• Last Job Failed In Flow Results in Flow Parent not being called #1333
• Parent job does not execute when sharing dependency with another job #1021
• Parent job does not execute when child job fails #800

[roggervalf]

hi @nggonzalez sorry for the delay, so trying to answer your questions:

Do flow child jobs with failParentOnFailure = true retry before failing the parent or do they immediately fail the parent on the first failure?

when a child job has backoff strategy, before the last retry, this child will not get the failed state, it will be moved to delayed state, this image could help on understand the different paths on our states https://docs.bullmq.io/guide/architecture. In the last retry, if the child fails again, it will be moved to failed state and the parent will be moved to failed.

Do flow parent jobs retry when they are failed by a child?

No, they are moved to failed directly, as those children that have failParentOnFailure = true are considered as hard dependencies.

And for your last question

Is there a way to run a flow parent when there is a failure in order to capture the error state

We should think more about it, as for now all children are consider hard dependencies, unless you have removeOnFail = true option in your children and failParentOnFailure = false, in that case the parent will be executed after the execution of all the children, but the link between the parent and their children will be lost.