removeOnComplete + Flow getChildrenValues

[rosslavery]

Hi there,

I'm looking to understand the interaction between the above-mentioned options/methods.

E.g. dispatching a child job with removeOnComplete: true, I would have expected the parent job that runs upon child completion to get back no data when calling job.getChildrenValues() since the child has been removed from Redis.

However upon testing, I can see the parent is still able to access the child returnValues🤔.

• Is this by design?
• And if so, how would someone purge child job return values from Redis to avoid infinitely eating up Redis memory?

[rosslavery]

As a follow-up to this...

To save redis space I updated my default job options to:

  removeOnComplete: {
    count: 5,
    age: 432000 // 5 days
  }

After this, my parent jobs were unable to retrieve any child values using job.getChildrenValues() which resulted in a large number of job failures. This would seem to contradict my earlier assertion that removeOnComplete: true has no impact on a parent's ability to access it's children values, even if they have been removed.

So now I'm more confused than before 😅

[manast]

The children's jobs need to exist so that getChildrenValues works correctly. So I guess the easy solution would be to manually delete the jobs after the parent has done the processing, as the last step. But I think this should work in a better way, as in either the children jobs are only deleted after the parent completes, or the children's jobs are deleted but the values are stored and deleted after the parent completes (I am leaning more on option 1). What do you think @roggervalf

[roggervalf]

yeah, I think the current behavior is expected as if for example we have a limited space in redis, removeOnComplete could remove the children records and the parent should keep the result value of those jobs (parents collect children results), if the parent has removeOnComplete, it should delete the children values after it is processed successfully (deleting a parent, delete children results as well). The way to delete the children values before parent is completed is using @manast suggestion:
"the easy solution would be to manually delete the jobs after the parent has done the processing, as the last step", removeJob method removes the result values from parent processed and unprocessed values