Skip to content

Commit

Permalink
fixup! add spark-operator-v2
Browse files Browse the repository at this point in the history
  • Loading branch information
ImpSy committed Dec 5, 2024
1 parent 4a0b3e6 commit 4cf27b9
Show file tree
Hide file tree
Showing 60 changed files with 26,993 additions and 8,877 deletions.
6 changes: 3 additions & 3 deletions charts/spark-operator/Chart.lock
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
dependencies:
- name: spark-operator
repository: ""
version: 1.1.20
digest: sha256:0beb33ae2ec41aff928f57b55d750f5134b96035908725744bb5c05a61eceb2b
generated: "2022-04-22T09:17:17.159186+02:00"
version: 2.0.2
digest: sha256:ec647402ce487be17941edc3508f3e41e2919833d666df2a956f9a3964944d05
generated: "2024-10-25T16:46:45.656964+02:00"
18 changes: 7 additions & 11 deletions charts/spark-operator/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,12 @@
apiVersion: v2
description: Spark Operator (b/g part)
name: spark-operator
version: 0.1.34
appVersion: v1beta2-1.3.4-3.1.1
dependencies:
- name: spark-operator
version: 1.1.20
# We use an unpacked manually managed sub chart / dependency chart - therefore we do not point to the chart repo
# To update the dependency chart:
# - uncomment this line and then run `helm dependency update charts/spark-operator`
# - unpack the .tgz file overriding the sub-chart, delete the .tgz file, and run `helm dependency update charts/spark-operator` again
# repository: https://googlecloudplatform.github.io/spark-on-k8s-operator
description: A Helm chart for Spark on Kubernetes operator.
version: 2.0.2
appVersion: 2.0.2
home: https://github.com/kubeflow/spark-operator
maintainers:
- name: thorsteinnth
email: thorsteinn.sigurdsson@netapp.com
dependencies:
- name: spark-operator
version: "2.0.2"
40 changes: 39 additions & 1 deletion charts/spark-operator/charts/spark-operator/.helmignore
Original file line number Diff line number Diff line change
@@ -1 +1,39 @@
ci/
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.

ci/
.helmignore

# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/

# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~

# Various IDEs
*.tmproj
.project
.idea/
.vscode/

# MacOS
.DS_Store

# helm-unittest
tests
.debug
__snapshot__

# helm-docs
README.md.gotmpl
44 changes: 36 additions & 8 deletions charts/spark-operator/charts/spark-operator/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,39 @@
#
# Copyright 2024 The Kubeflow authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

apiVersion: v2
appVersion: v1beta2-1.3.4-3.1.1
description: A Helm chart for Spark on Kubernetes operator
home: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator

name: spark-operator

description: A Helm chart for Spark on Kubernetes operator.

version: 2.0.2

appVersion: 2.0.2

keywords:
- spark
- apache spark
- big data

home: https://github.com/kubeflow/spark-operator

maintainers:
- email: yuchaoran2011@gmail.com
name: yuchaoran2011
name: spark-operator
version: 1.1.20
- name: yuchaoran2011
email: yuchaoran2011@gmail.com
url: https://github.com/yuchaoran2011
- name: ChenYi015
email: github@chenyicn.net
url: https://github.com/ChenYi015
218 changes: 128 additions & 90 deletions charts/spark-operator/charts/spark-operator/README.md

Large diffs are not rendered by default.

68 changes: 33 additions & 35 deletions charts/spark-operator/charts/spark-operator/README.md.gotmpl
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
{{ template "chart.header" . }}

{{ template "chart.deprecationWarning" . }}

{{ template "chart.badgesSection" . }}

{{ template "chart.description" . }}

{{ template "chart.homepageLine" . }}

## Introduction

This chart bootstraps a [Kubernetes Operator for Apache Spark]({{template "chart.homepage" . }}) deployment using the [Helm](https://helm.sh) package manager.
Expand All @@ -19,61 +25,53 @@ The previous `spark-operator` Helm chart hosted at [helm/charts](https://github.
- Previous versions of the Helm chart have not been migrated, and the version has been set to `1.0.0` at the onset. If you are looking for old versions of the chart, it's best to run `helm pull incubator/sparkoperator --version <your-version>` until you are ready to move to this repository's version.
- Several configuration properties have been changed, carefully review the [values](#values) section below to make sure you're aligned with the new values.

## Installing the chart
## Usage

### Add Helm Repo

```shell
helm repo add spark-operator https://kubeflow.github.io/spark-operator

helm repo update
```

See [helm repo](https://helm.sh/docs/helm/helm_repo) for command documentation.

$ helm repo add spark-operator https://googlecloudplatform.github.io/spark-on-k8s-operator
### Install the chart

$ helm install my-release spark-operator/spark-operator
```shell
helm install [RELEASE_NAME] spark-operator/spark-operator
```

This will create a release of `spark-operator` in the default namespace. To install in a different one:
For example, if you want to create a release with name `spark-operator` in the `spark-operator` namespace:

```shell
$ helm install -n spark my-release spark-operator/spark-operator
helm install spark-operator spark-operator/spark-operator \
--namespace spark-operator \
--create-namespace
```

Note that `helm` will fail to install if the namespace doesn't exist. Either create the namespace beforehand or pass the `--create-namespace` flag to the `helm install` command.
Note that by passing the `--create-namespace` flag to the `helm install` command, `helm` will create the release namespace if it does not exist.

## Uninstalling the chart
See [helm install](https://helm.sh/docs/helm/helm_install) for command documentation.

To uninstall `my-release`:
### Upgrade the chart

```shell
$ helm uninstall my-release
helm upgrade [RELEASE_NAME] spark-operator/spark-operator [flags]
```

The command removes all the Kubernetes components associated with the chart and deletes the release, except for the `crds`, those will have to be removed manually.

## Test the chart

Install [chart-testing cli](https://github.com/helm/chart-testing#installation)
See [helm upgrade](https://helm.sh/docs/helm/helm_upgrade) for command documentation.

In Mac OS, you can just:
### Uninstall the chart

```bash
pip install yamale
pip install yamllint
brew install chart-testing
```shell
helm uninstall [RELEASE_NAME]
```

Run ct lint and Verify `All charts linted successfully`

```bash
Chart version ok.
Validating /Users/chethanuk/Work/Github/Personal/spark-on-k8s-operator-1/charts/spark-operator-chart/Chart.yaml...
Validation success! 👍
Validating maintainers...
==> Linting charts/spark-operator-chart
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, 0 chart(s) failed
------------------------------------------------------------------------------------------------------------------------
✔︎ spark-operator => (version: "1.1.0", path: "charts/spark-operator-chart")
------------------------------------------------------------------------------------------------------------------------
All charts linted successfully
```
This removes all the Kubernetes resources associated with the chart and deletes the release, except for the `crds`, those will have to be removed manually.

See [helm uninstall](https://helm.sh/docs/helm/helm_uninstall) for command documentation.

{{ template "chart.valuesSection" . }}

Expand Down
2 changes: 2 additions & 0 deletions charts/spark-operator/charts/spark-operator/ci/ci-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
image:
tag: local
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
Loading

0 comments on commit 4cf27b9

Please sign in to comment.