Skip to content

Commit

Permalink
Merge pull request #113 from flanksource/updates
Browse files Browse the repository at this point in the history 
chore: doc updates
  • Loading branch information
@moshloop
moshloop authored Dec 19, 2023
2 parents 371e351 + 538d100 commit f4fe0b1
Show file tree
Hide file tree
Showing 8 changed files with 297 additions and 87 deletions.
147 changes: 124 additions & 23 deletions canary-checker/docs/concepts/metrics-exporter.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
title: Metrics Exporter
---



Canary checker can export custom metrics from any check type, replacing and/or consolidating multiple standaline Prometheus Exporters into a single exporter.

In the example below, exchange rates against the USD are exported by first calling an HTTP api and then using the values from the JSON response to create the metrics:
Expand Down Expand Up @@ -59,20 +57,122 @@ exchange_rate{from=USD, to=ILS} 3.849
exchange_rate_api 260.000
```

| Field | Description | Scheme | Required |
| --------- | ------------------------------------ | ------------------- | -------- |
## Stateful Metrics

Metrics can be generated from time based data, e.g. logs per minute, logins per second by using the output of one check execution as the input to the next.

```yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
name: "container-log-counts"
namespace: observability
# The schedule can be as short or as long as you want, the query will always search for log
# since the last query
schedule: "@every 5m"
http:
- name: container_log_volume
url: "https://elasticsearch/logstash-*/_search"
headers:
- name: Content-Type
value: application/json
templateBody: true
test:
# if no logs are found, fail the health check
expr: json.?aggregations.logs.doc_count.orValue(0) > 0
# query for log counts by namespace, container and pod that have been created since the last check
body: >-
{
"size": 0,
"aggs": {
"logs": {
"filter": {
"range": {
"@timestamp" : {
{{- if last_result.results.max }}
"gte": "{{ last_result.results.max }}"
{{- else }}
"gte": "now-5m"
{{- end }}
}
}
},
"aggs": {
"age": {
"max": {
"field": "@timestamp"
}
},
"labels": {
"multi_terms": {
"terms": [
{ "field": "kubernetes_namespace_name.keyword"},
{ "field": "kubernetes_container_name.keyword"},
{ "field": "kubernetes_pod_name.keyword"}
],
"size": 1000
}
}
}
}
}
}
transform:
# Save the maximum age for usage in subsequent queries and create a metric for each pair
expr: |
json.orValue(null) != null ?
[{
'detail': { 'max': string(json.?aggregations.logs.age.value_as_string.orValue(last_result().?results.max.orValue(time.Now()))) },
'metrics': json.?aggregations.logs.labels.buckets.orValue([]).map(k, {
'name': "namespace_log_count",
'type': "counter",
'value': double(k.doc_count),
'labels': {
"namespace": k.key[0],
"container": k.key[1],
"pod": k.key[2]
}
})
}].toJSON()
: '{}'
```
This snippet will retrieve the `last_result.results.max` value from the last execution ensuring data is not duplicated or missed
```go
"@timestamp" : {
{{- if last_result.results.max }}
"gte": "{{ last_result.results.max }}"
{{- else }}
"gte": "now-5m"
{{- end }}
}
```

The max value is saved in the `transform` section using:

```yaml
#...
'detail': { 'max': string(json.?aggregations.logs.age.value_as_string.orValue(last_result().?results.max.orValue(time.Now()))) },
#...
```

## Result Variables

| Field | Description | Scheme | Required |
| --------- | ------------------------------------ | ----------------- | -------- |
| `metrics` | Metrics to export from check results | [Metric](#metric) | |

### Metric

| Field | Description | Scheme | Required |
| -------- | ------------------------------------------------------ | ---------- | -------- |
| `name` | Name of the metric | `string` | Yes |
| `value` | An expression to derive the metric value from | `Expression` | Yes |
| `type` | Prometheus Metric Type (counter, gauge or histogram) | `string` | Yes |
| Field | Description | Scheme | Required |
| -------- | ------------------------------------------------------ | ----------------- | -------- |
| `name` | Name of the metric | `string` | Yes |
| `value` | An expression to derive the metric value from | `Expression` | Yes |
| `type` | Prometheus Metric Type (counter, gauge or histogram) | `string` | Yes |
| `labels` | Labels for prometheus metric (values can be templated) | [[]Label](#label) | |

#### Label
### Label

| Field | Description | Scheme | Required |
| ----------- | ------------------------------------------------------ | ------------------- | -------- |
Expand All @@ -83,17 +183,18 @@ exchange_rate_api 260.000

Expresions can make use of the following variables:

#### **Expression Variables**

| Fields | Description | Scheme |
| ------------------- | ------------------------------------------ | ----------------------------------------- |
| **`result`** | Check Result | See result variables section of the check |
| `check.name` | Check name | `string` |
| `check.description` | Check description | `string` |
| `check.labels` | Dynamic labels attached to the check | `map[string]string` |
| `check.endpoint` | Endpoint (usally a URL) | `string` |
| `check.duration` | Duration in milliseconds | `int64` |
| `canary.name` | Canary name | `string` |
| `canary.namespace` | Canary namespace | `string` |
| `canary.labels` | Labels attached to the canary CRD (if any) | `map[string]string` |
### **Expression Variables**

| Fields | Description | Scheme |
| ------------------------- | ------------------------------------------ | ----------------------------------------- |
| **`result`** | Check Result | See result variables section of the check |
| **`last_result.results`** | The last result | |
| `check.name` | Check name | `string` |
| `check.description` | Check description | `string` |
| `check.labels` | Dynamic labels attached to the check | `map[string]string` |
| `check.endpoint` | Endpoint (usally a URL) | `string` |
| `check.duration` | Duration in milliseconds | `int64` |
| `canary.name` | Canary name | `string` |
| `canary.namespace` | Canary namespace | `string` |
| `canary.labels` | Labels attached to the canary CRD (if any) | `map[string]string` |

133 changes: 111 additions & 22 deletions ...y-checker/docs/concepts/authentication.md → ...hecker/docs/concepts/secret-management.md
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Authentication
# Secret Management

Canary checker uses the Kubernetes ValuesFrom pattern to retrieve sensitive values like usernames, password and access keys.

Whenever a field uses the `EnvVar` object type you have the option of specifying the value in 3 ways:
Whenever a field uses the `EnvVar` object type you have the option of specifying the value in multiple ways.

## EnvVar
1. Statically in the `value`
1. Via a Kubernetes Config Map via the `configMapKeyRef`
1. Via a Kubernetes Secret via the `secretKeyRef`
1. From a value of a deployed helm chart using `helmRef`
1. From a service account using `serviceAccount`

1. Statically in the `value` field
1. Via a Kubernetes Config Map via the `configMapKeyRef` field
1. Via a Kubernetes Secret via the `secretKeyRef` field

### Static Values
## Static Values

Using a HTTP health check as an example for static values:

Expand All @@ -30,7 +30,7 @@ spec:
value: world
```
### Kubernetes Configmaps
## Configmaps
To use a configmap, we first need to create the configmap:
Expand Down Expand Up @@ -60,7 +60,7 @@ spec:
key: pass
```
### Kubernetes Secrets
## Secrets
To use a secret, first we create the secret:
Expand All @@ -75,22 +75,111 @@ metadata:
name: http-basic-auth-configmap
spec:
http:
- url: https://httpbin.org/basic-auth/hello/world
responseCodes: [200]
authentication:
username:
- url: https://httpbin.demo.aws.flanksource.com/basic-auth/hello/world
username:
valueFrom:
secretKeyRef:
name: basic-auth
key: user
password:
valueFrom:
secretKeyRef:
name: basic-auth
key: pass
```
## Helm Values
To use a secret, first we deploy a helm chart
```bash
helm install podinfo podinfo/podinfo -n podinfo --set ingress.enabled=true
```

```yaml title="http-from-helm.yaml"
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
name: http-from-helm
spec:
http:
- env:
- name: url
valueFrom:
secretKeyRef:
name: basic-auth
key: user
password:
helmRef:
name: podinfo
key: .ingress.hosts[0].host

url: $(url)
```
## Service Accounts
Checks can use service accounts for authentication with external services that have existing trust established
```yaml title="http-service-accounts.yaml"
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
name: http-basic-auth-configmap
spec:
http:

interval: 30
http:
- name: vault-example-sre
description: "HashiCorp Vault functionality check."
url: https://vault.example/v1/auth/kubernetes/login
env:
- name: TOKEN
valueFrom:
secretKeyRef:
name: basic-auth
key: pass
serviceAccount: default-account
templateBody: true
body: |
{
"jwt": "$(TOKEN)",
"role": "example-role"
}
```
:::note
For service account token issuing the canary-checker service account `canary-checker-sa` will need to be granted permissions to issue tokens using:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: canary-checker-sa-issuing-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: canary-checker-sa-issuing
subjects:
- kind: ServiceAccount
name: canary-checker-sa
namespace: canary-checker
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: canary-checker-sa-issuing
rules:
- apiGroups: [""]
resources:
- "serviceaccounts/token"
- "serviceaccounts"
verbs:
- "create"
- "get
```

### Recommendations
:::

# Recommendations

Kubernetes Secrets are, by default, stored unencrypted in the API server's underlying data store (etcd). Anyone with API access can retrieve or modify a Secret, and so can anyone with access to etcd. With this in mind, it is recommended to implement some level of security to prevent unauthorized access to your Kubernetes secrets.
You may consider the following for your encryption and security needs:
Expand Down
23 changes: 15 additions & 8 deletions canary-checker/docs/helm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ image: /static/img/icons/helm.svg

The recommended method for installing Canary Checker is using [helm](https://helm.sh/)

### 1. Add the Flanksource helm repository
## 1. Add the Flanksource helm repository

```bash
helm repo add flanksource https://flanksource.github.io/charts
helm repo update
```

### 2. Deploy Canary Checker using Helm
## 2. Deploy Canary Checker using Helm

To install into a new `canary-checker` namespace, run

Expand Down Expand Up @@ -83,11 +83,18 @@ kubectl port-forward svc/canary-checker-ui 8080:80

[http://localhost:8080](http://localhost:8080)

The canary checker itself only presents an API. To view the data graphically, the Flanksource UI is required, and is installed by default. The UI should be configured to allow external access to via ingress

| ---------------------------------- | ------------------------------------------------------------ |
| flanksource-ui.ingress.host | URL at which the UI will be accessed |
| flanksource-ui.ingress.annotations | Map of annotations required by the ingress controller or certificate issuer |
| flanksource-ui.ingress.tls | Map of configuration options for TLS |
To deploy an ingress for the dashboard, update the `values.yaml`:

More details regarding ingress configuration can be found in the [kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/)
```yaml
flanksource-ui:
enabled: true
ingress:
annotations:
kubernetes.io/tls-acme: "true"
host: <DOMAIN>
tls:
- hosts:
- <DOMAIN>
secretName: ingress-tls
```
3 changes: 2 additions & 1 deletion canary-checker/docusaurus.config.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ const config = {
customFields: {
oss: process.env.CANARY_CHECKER_OSS === 'true',
links: {
"authentication": '/concepts/authentication'
"authentication": '/concepts/secret-management',
"connection": '/concepts/authentication'
}
},
markdown: {
Expand Down
Loading

0 comments on commit f4fe0b1

Please sign in to comment.