More scheduling / delay support in Job Queues

[tyteen4a03]

It would be great to allow job queues to have more customization options related to time or scheduling

• Queue with delay: add a delay parameter to specify that the task should only run at least x seconds after being queued.
• Cron schedule for tasks: Allow tasks and workflows to be scheduled as cronjobs directly within Payload (when running in cron mode, for example)

[wkentdag]

+1, this would be very helpful for my scheduler plugin when I upgrade it for v3 https://github.com/wkentdag/payload-plugin-scheduler

[nathanclevenger]

in addition to allowing Workflows/Queues to pause/wait/sleep for a certain time period, or until a specified point in time, most of the other Workflow execution engines also provide a mechanism to wait for an event before continuing. Example use cases would be things like waiting for an invoice to be paid, or for a user to respond to a slack notification with a yes/no button.

Reference:
https://upstash.com/docs/workflow/basics/context#context-waitforevent
https://www.inngest.com/docs/features/inngest-functions/steps-workflows/wait-for-event

[DanRibbens]

I agree on the time based control, but I'm not understanding the idea of having an additional trigger to wait for an event.

I'm thinking the event itself is what should create the job then.

Why wouldn't you just create jobs when all conditions are met in your application logic?

[tyteen4a03]

I'm thinking the event itself is what should create the job then.

I am guessing this will get complicated when more and more events are introduced.

[nathanclevenger]

@DanRibbens in our use cases, we have many workflows where there are conditions that require the external event, and as such, it's not really feasible to create different jobs/workflows for each of those conditional permutations.

For example:

• If a customer's credit score is below a certain threshold, then we need to notify them to provide proof of income, and then wait for them to provide that proof of income, before the next step of the vehicle purchase workflow can continue.
• If a down payment is part of the deal, and only if a down payment is part of the deal, we need to send an invoice and wait for the payment of that invoice before processing of the workflow can continue
• We have various AI completions and LLM-as-judge evals, where if something is questionable, but only occasionally not required normally, then we send a slack notification and need to wait for an approve/deny response from the human in slack.

We absolutely love payload and it is the basis of our Auto Commerce Platform, and this new Workflow/Task/Queue capability is incredible ... but we cannot migrate our workflows into Payload from Upstash/Inngest until wait(interval), waitUntil(dateTime), and waitUntilEvent(event, id) are implemented.