From 81c02681130ff8d7b432306e19d5ae3f0a6de28a Mon Sep 17 00:00:00 2001 From: Vibhor Dabas Date: Fri, 12 Jan 2024 13:00:36 +0530 Subject: [PATCH] fix: write catalog landing page from a customer pov --- mission-control/docs/config-db/overview.md | 23 +++-- .../docs/config-db/scrapers/kubernetes.md | 93 +++++++++++++++++++ 2 files changed, 104 insertions(+), 12 deletions(-) diff --git a/mission-control/docs/config-db/overview.md b/mission-control/docs/config-db/overview.md index f3242aea..875a10ab 100644 --- a/mission-control/docs/config-db/overview.md +++ b/mission-control/docs/config-db/overview.md @@ -1,12 +1,18 @@ # Overview -![config db](../images/config-db.svg) +**Catalog** enhances configuration management by providing a clear view and search function for tracking changes across various dimensions such as nodes, zones, environments, applications, and technologies. This utility is enabled by a JSON-based database that regularly collects configuration data from multiple sources. The data is organized into a searchable JSON tree, facilitating efficient comparison and analysis across different environments. + +A configuration summary is shown below: -`Config DB` is a JSON-based configuration management database. It enables you to scrape configuration from several sources on an ongoing basis and navigate that configuration in an easy-to-navigate and search JSON tree. +![](../images/config-db.png) -By doing this, `Config DB` enables you to view and search the change history of your configuration across multiple dimensions _(node, zone, environment, application, technology, etc...)_ as well as compare and view the differences between configurations across environments. +Each configuration has: + +- **Configuration** - Normally JSON, but XML and properties files are also available +- **Insights** - Security, cost, performance, and other recommendations from scanners including AWS Trusted Advisor, AWS Config rules, etc. +- **Changes** - Either change directly on the config _(recorded as diff change type)_ or changes identified via AWS Cloudtrail, etc. -It is able to scan multiple configuration sources including +You can scan multiple configuration sources including: - [AWS Cloud Resources](./scrapers/aws.md) - [Azure Devops](./scrapers/azure-devops.md) - Azure Devops Pipeline runs @@ -16,14 +22,7 @@ It is able to scan multiple configuration sources including - [SQL](./scrapers/sql.md) - Data available via queries on MySQL, SQL Server, and Postgres databases - [Trivy](./scrapers/trivy.md) - Security scanning of Kubernetes clusters -Each configuration has: - -- **Configuration** - Normally JSON, but XML and properties files are also available -- **Insights** - Security, cost, performance, and other recommendations from scanners including AWS Trusted Advisor, AWS Config rules, etc... -- **Changes** - Either change directly on the config _(recorded as diff change type)_ or changes identified via AWS Cloudtrail, etc... - +![config db](../images/config-db.svg) -A configuration summary is shown below: -![](../images/config-db.png) diff --git a/mission-control/docs/config-db/scrapers/kubernetes.md b/mission-control/docs/config-db/scrapers/kubernetes.md index 9103ef6e..e15651ed 100644 --- a/mission-control/docs/config-db/scrapers/kubernetes.md +++ b/mission-control/docs/config-db/scrapers/kubernetes.md @@ -2,6 +2,66 @@ The `kubernetes` config type scrapes the configurations of your Kubernetes resources as specified with the fields; `namespace`, `selector`, `fieldSelector` and more. +Create a ScrapeConfig Custom Resource with the following manifest + +```yaml +apiVersion: configs.flanksource.com/v1 +kind: ScrapeConfig +metadata: + name: kubernetes-scraper +spec: + kubernetes: + - clusterName: local-kind-cluster + exclusions: + - Secret + - ReplicaSet + - APIService + - PodMetrics + - NodeMetrics + - endpoints.discovery.k8s.io + - endpointslices.discovery.k8s.io + - leases.coordination.k8s.io + - podmetrics.metrics.k8s.io + - nodemetrics.metrics.k8s.io + - customresourcedefinition + - controllerrevision + - certificaterequest + - orders.acme.cert-manager.io + relationships: + - kind: + expr: "has(spec.claimRef) ? spec.claimRef.kind : ''" + name: + expr: "has(spec.claimRef) ? spec.claimRef.name : ''" + namespace: + expr: "has(spec.claimRef) ? spec.claimRef.namespace : ''" + - kind: + value: Kustomization + name: + label: kustomize.toolkit.fluxcd.io/name + namespace: + label: kustomize.toolkit.fluxcd.io/namespace + - kind: + value: HelmRelease + name: + label: helm.toolkit.fluxcd.io/name + namespace: + label: helm.toolkit.fluxcd.io/namespace + event: + exclusions: + - SuccessfulCreate + - Created + - DNSConfigForming + severityKeywords: + error: + - failed + - error + warn: + - backoff + - nodeoutofmemory +``` + +Or from the UI add the spec: + ```yaml kubernetes: - clusterName: local-kind-cluster @@ -9,6 +69,8 @@ kubernetes: - Secret - ReplicaSet - APIService + - PodMetrics + - NodeMetrics - endpoints.discovery.k8s.io - endpointslices.discovery.k8s.io - leases.coordination.k8s.io @@ -30,6 +92,25 @@ kubernetes: warn: - backoff - nodeoutofmemory + relationships: + - kind: + expr: "has(spec.claimRef) ? spec.claimRef.kind : ''" + name: + expr: "has(spec.claimRef) ? spec.claimRef.name : ''" + namespace: + expr: "has(spec.claimRef) ? spec.claimRef.namespace : ''" + - kind: + value: Kustomization + name: + label: kustomize.toolkit.fluxcd.io/name + namespace: + label: kustomize.toolkit.fluxcd.io/namespace + - kind: + value: HelmRelease + name: + label: helm.toolkit.fluxcd.io/name + namespace: + label: helm.toolkit.fluxcd.io/namespace ``` ### Kubernetes @@ -57,6 +138,18 @@ kubernetes: | `exclusions` | Specify Kubernetes resources to be excluded from scraping | `[]string` | | | **`kubeconfig`** | Specify kubeconfig for access to your Kubernetes Cluster | [`kommons.EnvVar`](https://pkg.go.dev/github.com/flanksource/kommons#EnvVar) | yes | | `event` | Specify configuration to handle Kubernetes events. See [**KubernetesEvent**](#kubernetesevent) | [`KubernetesEvent`](#kubernetesevent) | yes | +| `relationships` | Helps the user to understand relationships amongst components, configurations and health checks. | [`Relationships`](#Relationships) | | + + +### Relationships + +Checks if a kubernetes source has the labels specified in `name` and `namespace` fields. Then based on the labels it looks up the matching resource config specified in `kind` and adds a relationship to it. + +| Field | Description | Scheme | Required | +| ------------------ | ------------------------------------------------------------------------------------------ | --------------------------------------- | -------- | +| `name` | The label with the name of the relative | `string` | Yes | +| `namespace` | The label with the namespace of the relative | `string` | Yes | +| `kind` | The resource type of the relative | `string` | Yes | ### KubernetesEvent