Skip to content

Commit

Permalink
support for Dynamic Resource Allocation doc update
Browse files Browse the repository at this point in the history
Signed-off-by: Moshe Levi <moshele@nvidia.com>
  • Loading branch information
moshe010 committed Nov 6, 2023
1 parent 49362e3 commit b8cf2e5
Showing 1 changed file with 117 additions and 4 deletions.
121 changes: 117 additions & 4 deletions docs/how-to-use.md
Original file line number Diff line number Diff line change
Expand Up @@ -511,7 +511,7 @@ spec:
EOF
```

We can then create a pod which uses the `default-route` key in the JSON formatted `k8s.v1.cni.cncf.io/networks` annotation.
We can then create a pod which uses the `default-route` key in the JSON formatted `k8s.v1.cni.cncf.io/networks` annotation.

```
cat <<EOF | kubectl create -f -
Expand All @@ -537,9 +537,9 @@ This will set `192.168.2.1` as the default route over the `net1` interface, such
```
kubectl exec -it samplepod -- ip route
default via 192.168.2.1 dev net1
10.244.0.0/24 dev eth0 proto kernel scope link src 10.244.0.169
10.244.0.0/16 via 10.244.0.1 dev eth0
default via 192.168.2.1 dev net1
10.244.0.0/24 dev eth0 proto kernel scope link src 10.244.0.169
10.244.0.0/16 via 10.244.0.1 dev eth0
```

## Entrypoint Parameters
Expand Down Expand Up @@ -634,3 +634,116 @@ Sometimes, you may wish to not have the entrypoint copy the binary file onto the
If you wish to have auto configuration use the `readinessindicatorfile` in the configuration, you can use the `--readiness-indicator-file` to express which file should be used as the readiness indicator.

--readiness-indicator-file=/path/to/file

### Run pod with network annotation and Dynamic Resource Allocation driver

Dynamic Resource Allocation is alternative mechanism to device plugin which allow to requests pod and container resources. The feature is alpha in k8s 1.27.

The following sections describe how to use DRA with multus and Nvidia DRA driver. Other DRA networking driver vendors should follow similar concepts to make use of multus DRA support.

#### Prerequisite

1. Kubernetes 1.27
2. Container Runtime with CDI support enabled
3. Kubernetes runtime-config=resource.k8s.io/v1alpha2
4. Kubernetes feature-gates=DynamicResourceAllocation=True,KubeletPodResourcesDynamicResources=true

#### Install DRA driver

The current example uses Nvidia DRA driver for networking. This DRA driver is not public avaialbe.

#### Create dynamic resource class with nvidia network DRA driver

The `ResourceClass` defines the resource pool of `sf-pool-1`.

```
# Execute following command at Kubernetes master
cat <<EOF | kubectl create -f -
apiVersion: resource.k8s.io/v1alpha2
kind: ResourceClass
metadata:
name: sf-pool-1
driverName: net.resource.nvidia.com
EOF
```

#### Create network attachment definition with resource name

The `k8s.v1.cni.cncf.io/resourceName` should match the `ResourceClass` name defined in the section above.
In this example it is `sf-pool-1`. Multus query the K8s PodResource API to fetch the `resourceClass` name and also
query the NetworkAttachmentDefinition `k8s.v1.cni.cncf.io/resourceName`. If both has the same name multus send the
CDI device name in the DeviceID argument.

##### NetworkAttachmentDefinition for ovn-kubernetes example:

Following command creates NetworkAttachmentDefinition. CNI config is in `config:` field.

```
# Execute following command at Kubernetes master
cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: default
annotations:
k8s.v1.cni.cncf.io/resourceName: sf-pool-1
spec:
config: '{
"cniVersion": "0.4.0",
"dns": {},
"ipam": {},
"logFile": "/var/log/ovn-kubernetes/ovn-k8s-cni-overlay.log",
"logLevel": "4",
"logfile-maxage": 5,
"logfile-maxbackups": 5,
"logfile-maxsize": 100,
"name": "ovn-kubernetes",
"type": "ovn-k8s-cni-overlay"
}'
EOF
```

#### Create DRA Resource Claim

Following command creates `ResourceClaim` `sf` which request resource from `ResourceClass` `sf-pool-1`.

```
# Execute following command at Kubernetes master
cat <<EOF | kubectl create -f -
apiVersion: resource.k8s.io/v1alpha2
kind: ResourceClaim
metadata:
namespace: default
name: sf
spec:
spec:
resourceClassName: sf-pool-1
EOF
```

#### Launch pod with DRA Resource Claim

Following command Launch a Pod with primiry network `default` and `ResourceClaim` `sf`.

```
apiVersion: v1
kind: Pod
metadata:
namespace: default
name: test-sf-claim
annotations:
v1.multus-cni.io/default-network: default
spec:
restartPolicy: Always
containers:
- name: with-resource
image: docker.io/library/ubuntu:22.04
command: ["/bin/sh", "-ec", "while :; do echo '.'; sleep 5 ; done"]
resources:
claims:
- name: resource
resourceClaims:
- name: resource
source:
resourceClaimName: sf
```

0 comments on commit b8cf2e5

Please sign in to comment.