Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify difference between scheduler flags #3

Open
ms705 opened this issue Jun 16, 2016 · 0 comments
Open

Clarify difference between scheduler flags #3

ms705 opened this issue Jun 16, 2016 · 0 comments

Comments

@ms705
Copy link
Contributor

ms705 commented Jun 16, 2016

We currently support (at least) two flags that control how workflows are scheduled:

  1. --use_dynamic_scheduler, which is a bool indicating whether the scheduler invokes DynamicScheduleDAG or ScheduleDAG.
  2. --use_heuristic, which is a bool indicating whether to use exhaustive search or the dynamic programming algorithm to find the system assignments (this manifests itself by calling SchedulerDynamic::ComputeOptimal or SchedulerDynamic::ComputeHeuristic).

IIRC, the difference between a dynamic and a non-dynamic scheduling call is whether Musketeer re-assesses its decisions after each job completion (@ICGog -- correct?).

The naming of things is a bit misleading here, for two reasons:

  1. The scheduler class is called SchedulerDynamic, but the --use_dynamic_scheduler flag controls which method in this class gets called, rather than whether a SchedulerDynamic or something else is used.
  2. Since our heuristic scheduler is based on dynamic programming, it is easy to assume that --use_dynamic_scheduler refers to the heuristic, which it does not.

Proposal: let's rename the flags, so that they're more intuitive.

  • --use_dynamic_scheduler becomes --continuously_reschedule
  • --use_heuristic becomes a --scheduling_algorithm flag that can take three values: "automatic" (use heuristic if >18 operators, optimal otherwise), "heuristic" and "optimal".

This came up while investigating bugs reported by @n1v0lg.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant