Skip to content

Commit

Permalink
Merge pull request #517 from danielvegamyhre/doc-update
Browse files Browse the repository at this point in the history 
Update docs for 0.5.0
  • Loading branch information
@k8s-ci-robot
k8s-ci-robot authored Apr 15, 2024
2 parents 9bf9079 + 8f0014d commit cb941fc
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 11 deletions.
12 changes: 6 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,17 @@ Take a look at the [concepts](https://jobset.sigs.k8s.io/docs/concepts/) page fo

- **Automatic headless service configuration and lifecycle management**: ML and HPC frameworks require a stable network endpoint for each worker in the distributed workload, and since pod IPs are dynamically assigned and can change between restarts, stable pod hostnames are required for distributed training on k8s, By default, JobSet uses [IndexedJobs](https://kubernetes.io/blog/2021/04/19/introducing-indexed-jobs/) to establish stable pod hostnames, and does automatic configuration and lifecycle management of the headless service to trigger DNS record creations and establish network connectivity via pod hostnames.

- **Configurable success policies**: JobSet has [configurable success policies](https://github.com/kubernetes-sigs/jobset/blob/v0.4.0/examples/simple/success-policy.yaml) which target specific ReplicatedJobs, with operators to target `Any` or `All` of their child jobs. For example, you can configure the JobSet to be marked complete if and only if all pods that are part of the “worker” ReplicatedJob are completed. This enables users to use their compute resources more efficiently, allowing a workload to be declared successful and release the resources for the next workload more quickly.
- **Configurable success policies**: JobSet has [configurable success policies](https://github.com/kubernetes-sigs/jobset/blob/v0.5.0/examples/simple/success-policy.yaml) which target specific ReplicatedJobs, with operators to target `Any` or `All` of their child jobs. For example, you can configure the JobSet to be marked complete if and only if all pods that are part of the “worker” ReplicatedJob are completed. This enables users to use their compute resources more efficiently, allowing a workload to be declared successful and release the resources for the next workload more quickly.

- **Configurable failure policies**: JobSet has [configurable failure policies](https://github.com/kubernetes-sigs/jobset/blob/v0.4.0/examples/simple/max-restarts.yaml) which allow the user to specify a maximum number of times the JobSet should be restarted in the event of a failure. If any job is marked failed, the entire JobSet will be recreated, allowing the workload to resume from the last checkpoint. When no failure policy is specified, if any job fails, the JobSet simply fails.
- **Configurable failure policies**: JobSet has [configurable failure policies](https://github.com/kubernetes-sigs/jobset/blob/v0.5.0/examples/simple/max-restarts.yaml) which allow the user to specify a maximum number of times the JobSet should be restarted in the event of a failure. If any job is marked failed, the entire JobSet will be recreated, allowing the workload to resume from the last checkpoint. When no failure policy is specified, if any job fails, the JobSet simply fails.

- **Exclusive Placement Per Topology Domain**: JobSet includes an [annotation](https://github.com/kubernetes-sigs/jobset/blob/1ae6c0c039c21d29083de38ae70d13c2c8ec613f/examples/simple/exclusive-placement.yaml#L6) which can be set by the user, specifying that there should be a 1:1 mapping between child job and a particular topology domain, such as a datacenter rack or zone. This means that all the pods belonging to a child job will be colocated in the same topology domain, while pods from other jobs will not be allowed to run within this domain. This gives the child job exclusive access to computer resources in this domain. You can run this [example](https://github.com/kubernetes-sigs/jobset/blob/v0.4.0/examples/simple/exclusive-placement.yaml) yourself to see how exclusive placement works.
- **Exclusive Placement Per Topology Domain**: JobSet includes an [annotation](https://github.com/kubernetes-sigs/jobset/blob/1ae6c0c039c21d29083de38ae70d13c2c8ec613f/examples/simple/exclusive-placement.yaml#L6) which can be set by the user, specifying that there should be a 1:1 mapping between child job and a particular topology domain, such as a datacenter rack or zone. This means that all the pods belonging to a child job will be colocated in the same topology domain, while pods from other jobs will not be allowed to run within this domain. This gives the child job exclusive access to computer resources in this domain. You can run this [example](https://github.com/kubernetes-sigs/jobset/blob/v0.5.0/examples/simple/exclusive-placement.yaml) yourself to see how exclusive placement works.

- **Fast failure recovery**: JobSet recovers from failures by recreating all the child Jobs. When scheduling constraints such as exclusive Job placement are used, fast failure recovery at scale can become challenging. As of JobSet v0.3.0, JobSet uses a designed such that it minimizes impact on scheduling throughput. We have benchmarked scheduling throughput during failure recovery at 290 pods/second at a 15k node scale.

- **Startup Sequencing**: As of JobSet v0.4.0 users can configure a [startup order](https://github.com/kubernetes-sigs/jobset/blob/v0.4.0/examples/startup-policy/startup-driver-ready.yaml) for the ReplicatedJobs in a JobSet. This enables support for patterns like the “leader-worker” paradigm, where the leader must be running before the workers should start up and connect to it.
- **Startup Sequencing**: As of JobSet v0.5.0 users can configure a [startup order](https://github.com/kubernetes-sigs/jobset/blob/v0.5.0/examples/startup-policy/startup-driver-ready.yaml) for the ReplicatedJobs in a JobSet. This enables support for patterns like the “leader-worker” paradigm, where the leader must be running before the workers should start up and connect to it.

- **Integration with Kueue**: Use JobSet v0.2.3+ and [Kueue](https://kueue.sigs.k8s.io/) v0.4.0+ to oversubscribe your cluster with JobSet workloads, placing them in queue which supports multi-tenancy, resource sharing and more. See [Kueue documentation](https://kueue.sigs.k8s.io/) for more details on the benefits of managing JobSet workloads via Kueue.
- **Integration with Kueue**: Use JobSet v0.2.3+ and [Kueue](https://kueue.sigs.k8s.io/) v0.5.0+ to oversubscribe your cluster with JobSet workloads, placing them in queue which supports multi-tenancy, resource sharing and more. See [Kueue documentation](https://kueue.sigs.k8s.io/) for more details on the benefits of managing JobSet workloads via Kueue.

## Production Readiness status

Expand All @@ -55,7 +55,7 @@ Take a look at the [concepts](https://jobset.sigs.k8s.io/docs/concepts/) page fo
To install the latest release of JobSet in your cluster, run the following command:

```shell
kubectl apply --server-side -f https://github.com/kubernetes-sigs/jobset/releases/download/v0.4.0/manifests.yaml
kubectl apply --server-side -f https://github.com/kubernetes-sigs/jobset/releases/download/v0.5.0/manifests.yaml
```

The controller runs in the `jobset-system` namespace.
Expand Down
10 changes: 5 additions & 5 deletions site/content/en/docs/installation/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ machine type for your nodes.**
To install a released version of Jobset in your cluster, run the following command:

```shell
VERSION=v0.4.0
VERSION=v0.5.0
kubectl apply --server-side -f https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/manifests.yaml
```

Expand All @@ -58,7 +58,7 @@ If you are using [prometheus-operator](https://github.com/prometheus-operator/pr
to scrape metrics from jobset components, run the following command:

```shell
VERSION=v0.4.0
VERSION=v0.5.0
kubectl apply -f https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/prometheus.yaml
```

Expand All @@ -71,7 +71,7 @@ can be scraped without performing this step.
To uninstall a released version of JobSet from your cluster, run the following command:

```shell
VERSION=v0.4.0
VERSION=v0.5.0
kubectl delete -f https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/manifests.yaml
```

Expand All @@ -83,14 +83,14 @@ kubectl delete -f https://github.com/kubernetes-sigs/jobset/releases/download/$V
<!-- <\!-- 1. Download the release's `manifests.yaml` file: -\-> -->

<!-- <\!-- ```shell -\-> -->
<!-- <\!-- VERSION=v0.4.0 -\-> -->
<!-- <\!-- VERSION=v0.5.0 -\-> -->
<!-- <\!-- wget https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/manifests.yaml -\-> -->
<!-- <\!-- ``` -\-> -->
<!-- <\!-- 2. With an editor of your preference, open `manifests.yaml`. -\-> -->
<!-- <\!-- 3. In the `jobset-manager-config` ConfigMap manifest, edit the -\-> -->
<!-- <\!-- `controller_manager_config.yaml` data entry. The entry represents -\-> -->
<!-- <\!-- the default JobSet Configuration -\-> -->
<!-- <\!-- struct ([v1alpha2@v0.4.0](https://pkg.go.dev/sigs.k8s.io/jobset@v0.4.0/apis/config/v1alpha2#Configuration)). -\-> -->
<!-- <\!-- struct ([v1alpha2@v0.5.0](https://pkg.go.dev/sigs.k8s.io/jobset@v0.5.0/apis/config/v1alpha2#Configuration)). -\-> -->
<!-- <\!-- The contents of the ConfigMap are similar to the following: -\-> -->


Expand Down

0 comments on commit cb941fc

Please sign in to comment.