You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We currently support (at least) two flags that control how workflows are scheduled:
--use_dynamic_scheduler, which is a bool indicating whether the scheduler invokes DynamicScheduleDAG or ScheduleDAG.
--use_heuristic, which is a bool indicating whether to use exhaustive search or the dynamic programming algorithm to find the system assignments (this manifests itself by calling SchedulerDynamic::ComputeOptimal or SchedulerDynamic::ComputeHeuristic).
IIRC, the difference between a dynamic and a non-dynamic scheduling call is whether Musketeer re-assesses its decisions after each job completion (@ICGog -- correct?).
The naming of things is a bit misleading here, for two reasons:
The scheduler class is called SchedulerDynamic, but the --use_dynamic_scheduler flag controls which method in this class gets called, rather than whether a SchedulerDynamic or something else is used.
Since our heuristic scheduler is based on dynamic programming, it is easy to assume that --use_dynamic_scheduler refers to the heuristic, which it does not.
Proposal: let's rename the flags, so that they're more intuitive.
--use_heuristic becomes a --scheduling_algorithm flag that can take three values: "automatic" (use heuristic if >18 operators, optimal otherwise), "heuristic" and "optimal".
This came up while investigating bugs reported by @n1v0lg.
The text was updated successfully, but these errors were encountered:
We currently support (at least) two flags that control how workflows are scheduled:
--use_dynamic_scheduler
, which is abool
indicating whether the scheduler invokesDynamicScheduleDAG
orScheduleDAG
.--use_heuristic
, which is abool
indicating whether to use exhaustive search or the dynamic programming algorithm to find the system assignments (this manifests itself by callingSchedulerDynamic::ComputeOptimal
orSchedulerDynamic::ComputeHeuristic
).IIRC, the difference between a dynamic and a non-dynamic scheduling call is whether Musketeer re-assesses its decisions after each job completion (@ICGog -- correct?).
The naming of things is a bit misleading here, for two reasons:
SchedulerDynamic
, but the--use_dynamic_scheduler
flag controls which method in this class gets called, rather than whether aSchedulerDynamic
or something else is used.--use_dynamic_scheduler
refers to the heuristic, which it does not.Proposal: let's rename the flags, so that they're more intuitive.
--use_dynamic_scheduler
becomes--continuously_reschedule
--use_heuristic
becomes a--scheduling_algorithm
flag that can take three values: "automatic" (use heuristic if >18 operators, optimal otherwise), "heuristic" and "optimal".This came up while investigating bugs reported by @n1v0lg.
The text was updated successfully, but these errors were encountered: