Skip to content

Commit

Permalink
Update code to scale regarding n labels or annotations
Browse files Browse the repository at this point in the history
  • Loading branch information
christopherime committed Mar 12, 2023
1 parent d7b1e75 commit 2f492a4
Show file tree
Hide file tree
Showing 4 changed files with 74 additions and 114 deletions.
8 changes: 0 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,5 @@ build:
-o binary/$(IMAGE_NAME) \
./...

timestamp:
mkdir -p binary
go build -a \
-gcflags=all="-l -B" \
-ldflags="-w -s" \
-o binary/$(IMAGE_NAME)_$(shell date +"%Y-%m-%dT%H_%M") \
./...

clean:
rm -rf binary
47 changes: 0 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,6 @@ The `GetAllMetricsMetadata` function is then called to retrieve the metadata for

The `GetAllAlertingRules` function is then called to retrieve the alerting rules from the Prometheus instance. The function sends a GET request to the `/api/v1/rules` endpoint of the Prometheus API, parses the response and returns the rules in an `AlertingRulesResponseType` struct.

Finally, the `WriteMetricsMetadataToCSV` function is called to export the metadata for all the metrics to a CSV file named `metrics.csv` in the `output` directory.


### Diagram

```mermaid
Expand All @@ -124,50 +121,6 @@ sequenceDiagram
Tool-->>User: Export complete
```

### Functions

- `SetInsecureSSL()` This function sets the `InsecureSkipVerify` field of the TLS config to true, which allows the tool to ignore SSL certificate validation errors. This function is used in the `ApiCaller` function to allow connections to Prometheus instances with self-signed certificates.

- `ApiCaller(url string, method string, body io.Reader, headers map[string]string) ([]byte, int, error)` This function sends an HTTP request to a given URL with the specified method and body and returns the response body, status code and error (if any). This function is used to interact with the Prometheus API.

- `GetAllMetricsMetadata(prometheusURL string) MetricsMetadataResponseType` This function retrieves the metadata for all the metrics from a given Prometheus instance by sending a GET request to the `/api/v1/targets/metadata` endpoint of the Prometheus API. The function returns a `MetricsMetadataResponseType` struct containing the metadata for each metric.

- `GetAllAlertingRules(prometheusURL string) AlertingRulesResponseType` This function retrieves the alerting rules from a given Prometheus instance by sending a GET request to the `/api/v1/rules` endpoint of the Prometheus API. The function returns an `AlertingRulesResponseType` struct containing the rules.

- `WriteMetricsMetadataToCSV(metricsMetadata MetricsMetadataResponseType, filename string) error` This function writes the metadata for all the metrics to a CSV file named `metrics.csv` in the `output` directory. The function takes a `MetricsMetadataResponseType` struct as input and returns an error if the file cannot be created or if there is an error while writing the data.

### Types

- `MetricsResponseType`: This type represents the response from the `/api/v1/targets/metadata` endpoint of the Prometheus API. It contains a `Status` field with the status of the response and a `Data` field with an array of `MetricsNamesType` strings representing the names of the metrics.

- `MetricsNamesType`: This type is a string

- `MetricsMetadataResponseType`: This type represents the response from the /api/v1/targets/metadata endpoint of the Prometheus API when requesting metadata for all the metrics. It contains a Status field with the status of the response and a Data field with an array of MetricsMetadataType structs representing the metadata for each metric.

- `MetricsMetadataType`: This type represents the metadata for a single metric. It contains a Target field with the instance and job labels of the metric, a Metric field with the name of the metric, a Type field with the type of the metric, a Help field with the help string of the metric, and a Unit field with the unit of the metric.

- `TargetMetadataType`: This type represents the instance and job labels of a metric.

- `AlertingRulesResponseType`: This type represents the response from the /api/v1/rules endpoint of the Prometheus API when requesting alerting rules. It contains a Data field with an array of Groups structs representing the alerting rule groups.

- `RuleType`: This type represents an alerting rule. It contains an array of Alerts structs representing the active alerts for the rule, an Annotations field with the annotations of the rule, a Health field with the health status of the rule, a Labels field with the labels of the rule, a Name field with the name of the rule, a Query field with the query of the rule, a Type field with the type of the rule, a Duration field with the duration of the rule, a Groups field with the groups of the rule, and a LastExecuted field with the timestamp of the last execution of the rule.

- `RuleAnnotationsType`: This type represents the annotations of an alerting rule. It contains a Description field with the description of the rule and a Summary field with the summary of the rule.

- `RuleLabelsType`: This type represents the labels of an alerting rule. It contains a Severity field with the severity of the rule.

- `AlertType`: This type represents an active alert for an alerting rule. It contains an ActiveAt field with the timestamp of when the alert became active, an Annotations field with the annotations of the alert, a Labels field with the labels of the alert, a State field with the state of the alert, and a Value field with the value of the alert.

### Type Diagram

```mermaid
classDiagram
MetricsMetadataResponseType --|> MetricsResponseType
MetricsMetadataType --|> TargetMetadataType
AlertingRulesResponseType
RuleType --|> RuleAnnotationsType
```

## External libraries

- [charmbracelet/log](https://github.com/charmbracelet/log): Used to log messages to the console
Expand Down
83 changes: 54 additions & 29 deletions src/function.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,6 @@ func GetAllAlertingRules(prometheusURL string) AlertingRulesResponseType {
}

func WriteMetricsMetadataToCSV(metricsMetadata MetricsMetadataResponseType, filename string) error {

file, err := os.Create(filename)
if err != nil {
return err
Expand All @@ -114,17 +113,29 @@ func WriteMetricsMetadataToCSV(metricsMetadata MetricsMetadataResponseType, file

writer := csv.NewWriter(file)
defer writer.Flush()
writer.Write([]string{"instance", "job", "metric", "type", "help", "unit"})

for _, metadata := range metricsMetadata.Data {
instance := metadata.Target.Instance
job := metadata.Target.Job
metric := metadata.Metric
metricType := metadata.Type
help := metadata.Help
unit := metadata.Unit
// Generate the CSV header dynamically based on the keys in metricsMetadata.Data.Target
targetKeys := make([]string, 0, len(metricsMetadata.Data[0].Target))
for k := range metricsMetadata.Data[0].Target {
targetKeys = append(targetKeys, k)
}
header := append([]string{"metric", "type", "help", "unit"}, targetKeys...)
writer.Write(header)

writer.Write([]string{instance, job, metric, metricType, help, unit})
pattern := regexp.MustCompile(`\n`)
for _, metadata := range metricsMetadata.Data {
metric := pattern.ReplaceAllString(metadata.Metric, " ")
metricType := pattern.ReplaceAllString(metadata.Type, " ")
help := pattern.ReplaceAllString(metadata.Help, " ")
unit := pattern.ReplaceAllString(metadata.Unit, " ")

// Construct the row to write to the CSV
row := make([]string, 0, len(targetKeys)+4)
row = append(row, metric, metricType, help, unit)
for _, key := range targetKeys {
row = append(row, metadata.Target[key])
}
writer.Write(row)
}

return nil
Expand All @@ -139,39 +150,53 @@ func WriteAlertingRulesToCSV(alertingRules AlertingRulesResponseType, filename s

writer := csv.NewWriter(file)
defer writer.Flush()
writer.Write([]string{"instance", "alertname", "query", "summary", "description"})

// Create the regex pattern to match newline characters
pattern := regexp.MustCompile(`\n`)

// Get the label keys for the first rule (assuming all rules have the same keys)
labelKeys := make([]string, 0)
if len(alertingRules.Data.Groups) > 0 && len(alertingRules.Data.Groups[0].Rules) > 0 {
for k := range alertingRules.Data.Groups[0].Rules[0].Labels {
labelKeys = append(labelKeys, k)
// Generate the CSV header dynamically based on the label and annotation keys
labelKeys := make(map[string]bool)
annotationKeys := make(map[string]bool)
for _, group := range alertingRules.Data.Groups {
for _, rule := range group.Rules {
for k := range rule.Labels {
labelKeys[k] = true
}
for k := range rule.Annotations {
annotationKeys[k] = true
}
}
}
header := []string{"alertname", "query"}
for k := range labelKeys {
header = append(header, k)
}
for k := range annotationKeys {
header = append(header, k)
}
writer.Write(header)

// Create the regex pattern to match newline characters
pattern := regexp.MustCompile(`\n`)

for _, group := range alertingRules.Data.Groups {
for _, rule := range group.Rules {
instance := ""
if value, ok := rule.Labels["instance"]; ok {
instance = value
}
alertname := rule.Name
query := rule.Query
summary := pattern.ReplaceAllString(rule.Annotations.Summary, " ")
description := pattern.ReplaceAllString(rule.Annotations.Description, " ")

// Write the label values for each key
row := []string{instance, alertname, query, summary, description}
for _, key := range labelKeys {
// Write the label and annotation values for each key
row := []string{alertname, query}
for k := range labelKeys {
value := ""
if v, ok := rule.Labels[key]; ok {
if v, ok := rule.Labels[k]; ok {
value = v
}
row = append(row, value)
}
for k := range annotationKeys {
value := ""
if v, ok := rule.Annotations[k]; ok {
value = v
}
row = append(row, pattern.ReplaceAllString(value, " "))
}

writer.Write(row)
}
Expand Down
50 changes: 20 additions & 30 deletions src/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,11 @@ type MetricsMetadataResponseType struct {
}

type MetricsMetadataType struct {
Target struct {
Instance string `json:"instance"`
Job string `json:"job"`
} `json:"target"`
Metric string `json:"metric"`
Type string `json:"type"`
Help string `json:"help"`
Unit string `json:"unit"`
Target map[string]string `json:"target"`
Metric string `json:"metric"`
Type string `json:"type"`
Help string `json:"help"`
Unit string `json:"unit"`
}

type TargetMetadataType struct {
Expand All @@ -40,29 +37,22 @@ type AlertingRulesResponseType struct {
}

type RuleType struct {
Alerts []AlertType `json:"alerts"`
Annotations AnnotationsType `json:"annotations"`
Health string `json:"health"`
Labels RuleLabelsType `json:"labels"`
Name string `json:"name"`
Query string `json:"query"`
Type string `json:"type"`
Duration int `json:"duration"`
Groups []string `json:"groups"`
LastExecuted int `json:"lastExecuted"`
Alerts []AlertType `json:"alerts"`
Annotations map[string]string `json:"annotations"`
Health string `json:"health"`
Labels map[string]string `json:"labels"`
Name string `json:"name"`
Query string `json:"query"`
Type string `json:"type"`
Duration int `json:"duration"`
Groups []string `json:"groups"`
LastExecuted int `json:"lastExecuted"`
}

type AlertType struct {
ActiveAt time.Time `json:"activeAt"`
Annotations AnnotationsType `json:"annotations"`
Labels RuleLabelsType `json:"labels"`
State string `json:"state"`
Value string `json:"value"`
ActiveAt time.Time `json:"activeAt"`
Annotations map[string]string `json:"annotations"`
Labels map[string]string `json:"labels"`
State string `json:"state"`
Value string `json:"value"`
}

type AnnotationsType struct {
Description string `json:"description"`
Summary string `json:"summary"`
}

type RuleLabelsType map[string]string

0 comments on commit 2f492a4

Please sign in to comment.