Skip to content

Commit

Permalink
Merge branch 'main' into canary-checker-testing
Browse files Browse the repository at this point in the history
  • Loading branch information
@dabasvibhor
dabasvibhor committed Dec 26, 2023
2 parents e94eeb0 + 0c7d165 commit 33b9fd0
Show file tree
Hide file tree
Showing 7 changed files with 287 additions and 71 deletions.
90 changes: 90 additions & 0 deletions canary-checker/docs/concepts/artifacts.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Artifacts

Artifacts allow you to archive files generated by some checks to a file store of your choice.

## Filestores

- AWS S3
- Google Cloud Storage
- SFTP
- SMB
- Local Filesystem

## Setting up artifact store

One of the above filestores can be used as the global artifact store. To setup an artifact store, pass the connection name of the store using the `artifact-connection` flag. If the artifact connection isn't setup, the artifacts are simply ignored.

Example:

```bash
canary-checker --artifact-connection='connection://sftp/artifacts'
```

## Archiving artifacts

The following checks support archiving artifacts

- [Exec](../reference/exec)
- [Junit](../reference/junit)

The only configuration required is to provide the path(s) of the artifacts generated by the check.

### Ex1. Archiving `/tmp/results/` directory

For the following script in an exec check

```
mkdir -p /tmp/results
curl -sL 'https://example.com' > /tmp/results/example.com
curl -sL 'https://flanksource.com' > /tmp/results/flanksource.com
```

one can provide the artifact paths as follows

```yaml title="archive-websites.yaml"
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
name: exec-artifact
spec:
interval: 30
exec:
- name: Download example.com and flanksource.com
script: |
mkdir -p /tmp/results
curl -sL 'https://example.com' > /tmp/results/example.com
curl -sL 'https://flanksource.com' > /tmp/results/flanksource.com
artifacts:
- path: /tmp/results/example.com
- path: /tmp/results/flanksource.com
```
or, use a glob as
```yaml
artifacts
- path: '/tmp/results/*.com'
```
### Ex2. Archiving artifacts from stdout/stderr
The path field accepts two special paths
- `/dev/stdout`
- `/dev/stderr`

```yaml title="archive-website.yaml"
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
name: exec-artifact
spec:
interval: 30
exec:
- name: Archive response of example.com
script: |
curl -sL 'https://example.com'
artifacts:
- path: /dev/stdout
- path: /dev/stderr
```
43 changes: 43 additions & 0 deletions canary-checker/docs/concepts/scaling.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Scaling Canaries up/down

Canaries can be scaled up or down just like other kubernetes resources. However, scaling to more than one replica is identical to having just one replica. In other words, scaling a canary can be thought of as a mechanism to turn on/off the canary.

## Example

1. List all the canaries

```bash
kubectl get canaries -o wide
# NAME REPLICAS INTERVAL STATUS LAST CHECK UPTIME 1H LATENCY 1H LAST TRANSITIONED ...
# folder-pass 1 300 Failed 34s 0/35 0% 16ms
# folder-pass-empty 1 300 Passed 34s 37/37 (100.0%) 0ms
# s3-bucket-pass 1 30 Failed 3s 0/358 0% 1s
```

2. Scale one of the canaries to 10 replicas

```bash
kubectl scale --replicas=10 canaries.canaries.flanksource.com folder-pass
# canary.canaries.flanksource.com/folder-pass scaled
```

```bash
kubectl get canaries folder-pass -o wide
# NAME REPLICAS INTERVAL STATUS LAST CHECK UPTIME 1H LATENCY 1H LAST TRANSITIONED MESSAGE ERROR
# folder-pass 10 300 Failed 3m13s 0/35 0% 16ms
```

3. Scale it down to 0 replicas

```bash
kubectl scale --replicas=0 canaries.canaries.flanksource.com folder-pass
# canary.canaries.flanksource.com/folder-pass scaled
```

```bash
kubectl get canaries folder-pass -o wide
# NAME REPLICAS INTERVAL STATUS LAST CHECK UPTIME 1H LATENCY 1H LAST TRANSITIONED MESSAGE ERROR
# folder-pass 0 300 Failed 3m13s 0/35 0% 16ms
```

This effectively stops the canary from running.
31 changes: 31 additions & 0 deletions canary-checker/docs/reference/connections.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Connections

### AWS Connection

| Field | Description | Type | Required |
| ---------------- | --------------- | ----------------------------------------------------------------------------- | -------- |
| `connectionName` | Connection name | `string` | |
| `accessKey` | Access key | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `secretKey` | Secret key | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `region` | Region | `string` | |
| `endpoint` | Endpoint | `string` | |
| `skipTLSVerify` | Skip TLS verify | `bool` | |
| `objectPath` | Object path | `string` | |
| `usePathStyle` | Use path style | `bool` | |

### GCP Connection

| Field | Description | Type | Required |
| ---------------- | --------------- | ----------------------------------------------------------------------------- | -------- |
| `connectionName` | Connection name | `string` | |
| `endpoint` | Endpoint | `string` | |
| `credentials` | Credentials | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |

### Azure Connection

| Field | Description | Type | Required |
| ---------------- | --------------- | ----------------------------------------------------------------------------- | -------- |
| `connectionName` | Connection name | `string` | |
| `clientID` | Client ID | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `clientSecret` | Client Secret | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `tenantID` | Tenant ID | `string` | |
72 changes: 54 additions & 18 deletions canary-checker/docs/reference/exec.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ title: Exec

<Standard />

Executes a bash (linux) or powershell (windows) script. The check is considered passing if the script exits with `0`
Executes a bash (linux) or powershell (windows) script. The check is considered passing if the script exits with `0`

```yaml
apiVersion: canaries.flanksource.com/v1
Expand All @@ -16,27 +16,63 @@ metadata:
spec:
interval: 30
exec:
- description: "exec dummy check"
script: |
echo "hello"
name: exec-pass-check
test:
expr: 'results.Stdout == "hello"'
- description: "exec dummy check"
script: |
echo "hello"
name: exec-pass-check
test:
expr: 'results.Stdout == "hello"'
```
| Field | Description | Scheme | Required |
| ----- | ----------- | ------ | -------- |
| **`script`** | Script can be a inline script or a path to a script that needs to be executed. On windows executed via Powershell and in darwin and linux executed using bash. | *string* | Yes |
| **`name`** | Name of the check, must be unique within the canary | `string` | Yes |
| `description` | Description for the check | `string` | |
| `icon` | Icon for overwriting default icon on the dashboard | `string` | |
| `labels` | Labels for check | `map[string]string` | |
| `test` | Evaluate whether a check is healthy | [`Expression`](../concepts/health-evaluation) | |
| `display` | Expression to change the formatting of the display | [`Expression`](../concepts/display-formatting) | |
| `transform` | Transform data from a check into multiple individual checks | [`Expression`](../concepts/transforms) | |
| `metrics` | Metrics to export from | [`[]Metrics`](../concepts/metrics-exporter) | |
| Field | Description | Scheme | Required |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | -------- |
| **`script`** | Script can be a inline script or a path to a script that needs to be executed. On windows executed via Powershell and in darwin and linux executed using bash. | _string_ | Yes |
| **`name`** | Name of the check, must be unique within the canary | `string` | Yes |
| `artifacts` | Specify what files/folders generated by the script needs to be archived | [`[]Artifact`](#artifact) | |
| `checkout` | Specify git repository to mount to the exec process | [`[]GitCheckout`](#git-checkout) | |
| `connections` | Setup connections for the script (eg: aws credentials) | [`ExecConnection`](#exec-connection) | |
| `env` | Setup environment variables that are accessible to the exec process | [`[]types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `namespace` | Namespace of the check | `string` | |
| `description` | Description for the check | `string` | |
| `icon` | Icon for overwriting default icon on the dashboard | `string` | |
| `labels` | Labels for check | `map[string]string` | |
| `metrics` | Metrics to export from | [`[]Metrics`](../concepts/metrics-exporter) | |
| `test` | Evaluate whether a check is healthy | [`Expression`](../concepts/health-evaluation) | |
| `display` | Expression to change the formatting of the display | [`Expression`](../concepts/display-formatting) | |
| `transform` | Transform data from a check into multiple individual checks | [`Expression`](../concepts/transforms) | |

:::tip Docker Image variants
See [image-variants](../concepts/image-variants)
:::

### Exec Connection

Exec connections allow you to specify credentials for a list of CLI tools that are needed by your scripts. Eg: You can specify the AWS connection name and the credential files along with the necessary environment variables
will be setup on the host running the script.

| Field | Description | Type | Required |
| ------- | ---------------- | ----------------------------------------------------------------- | -------- |
| `aws` | AWS connection | [`AWSConnection`](../reference/connections.md#aws-connection) | |
| `gcp` | GCP connection | [`GCPConnection`](../reference/connections.md#gcp-connection) | |
| `azure` | Azure connection | [`AzureConnection`](../reference/connections.md#azure-connection) | |

### Artifact

| Field | Description | Type | Required |
| ------ | ------------- | -------- | -------- |
| `path` | Path or glob. | `string` | `true` |

[Read more ...](../concepts/artifacts)

### Git Checkout

For authentication, either provide the connection name or the basic auth or the certificate.

| Field | Description | Type | Required |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | -------- |
| `url` | Git repository URL. | `string` | `true` |
| `connection` | Specify the connection name to use for git authentication (if required) | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `username` | Git auth username. | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `password` | Git auth password. | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `certificate` | Git auth certificate. | [`types.EnvVar`](https://pkg.go.dev/github.com/flanksource/duty/types#EnvVar) | |
| `destination` | Destination is the full path to where the contents of the URL should be downloaded to. If left empty, the sha256 hash of the URL will be used as the dir name. | `string` | |
Loading

0 comments on commit 33b9fd0

Please sign in to comment.