Use Gang Scheduling in ElasticJob of DLRover.

[workingloong]

For PyTorch elastic synchronous training jobs, the number of workers is typically set between min_nodes and max_nodes. If the number of nodes is less than min_nodes, the training iteration cannot start, with the initially launched workers occupying resources while waiting, leading to a waste of hardware capability. Gang Scheduling, on the other hand, will not launch the worker Pods until the number of available nodes in the cluster is at least min_nodes.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.

[github-actions]

This issue is being automatically closed due to inactivity.