Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provide-consumed-resources-from-prometheus #676

Merged
merged 35 commits into from
Oct 1, 2024
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
a3904b5
Added resources endpoint
satr Sep 9, 2024
2e91f35
Added prometheus client
satr Sep 11, 2024
abdb3d2
Added multiple prometheus queries
satr Sep 11, 2024
84ae9a5
Extracted prometheus metrics logic
satr Sep 11, 2024
46a26c2
Corrected min-max-avg values
satr Sep 11, 2024
f1a942a
Added comment and correct URL
satr Sep 11, 2024
2045c74
Added query arguments
satr Sep 11, 2024
23242e1
Added query arguments
satr Sep 12, 2024
d47f680
Added extra query arguments
satr Sep 12, 2024
9e380d7
Added extra query arguments
satr Sep 12, 2024
0601fff
Updated queries
satr Sep 13, 2024
292abae
Cleanup
satr Sep 13, 2024
5cb54d6
Cleanup
satr Sep 13, 2024
eacca9a
Extracted prometheus handler
satr Sep 13, 2024
ee63d27
Adding unit-tests
satr Sep 16, 2024
439677f
Adding unit-tests
satr Sep 16, 2024
b3482b0
Adding unit-tests
satr Sep 16, 2024
fc939dc
Merge remote-tracking branch 'origin/master' into provide-consumed-re…
satr Sep 17, 2024
6803ee5
Adding unit-tests
satr Sep 19, 2024
13cda76
Adding unit-tests
satr Sep 20, 2024
7b06569
Adding unit-tests
satr Sep 20, 2024
3da30b6
Merge remote-tracking branch 'origin/master' into provide-consumed-re…
satr Sep 20, 2024
30c2e10
Updated refs
satr Sep 20, 2024
d185ed7
Renamed var
satr Sep 20, 2024
72656e5
Send only raw values
satr Sep 24, 2024
25c56f2
Always remove zero values
satr Sep 24, 2024
c642e64
Updated objects
satr Sep 24, 2024
6b9bd1f
Tuned queries
satr Sep 25, 2024
1d5486d
Corrected summary logic
satr Sep 25, 2024
0b19cc5
Merge remote-tracking branch 'origin/master' into provide-consumed-re…
satr Sep 27, 2024
4b1b3ad
Updated versions
satr Sep 27, 2024
e7273d9
Fixed tests
satr Sep 30, 2024
ac2bbc7
Fixed tests
satr Sep 30, 2024
70e8c21
Delete file
satr Oct 1, 2024
8e4ae6b
Moved the method
satr Oct 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
"REQUIRE_APP_CONFIGURATION_ITEM": "true",
"REQUIRE_APP_AD_GROUPS": "true",
"RADIX_ENVIRONMENT":"qa",
"PROMETHEUS_URL":"http://localhost:9091",
"RADIX_APP":"radix-api",
"LOG_LEVEL":"info",
"LOG_PRETTY":"true"
Expand Down
2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ build: $(BINS)
mocks: bootstrap
mockgen -source ./api/buildstatus/models/buildstatus.go -destination ./api/test/mock/buildstatus_mock.go -package mock
mockgen -source ./api/deployments/deployment_handler.go -destination ./api/deployments/mock/deployment_handler_mock.go -package mock
mockgen -source ./api/metrics/prometheus_handler.go -destination ./api/metrics/mock/prometheus_handler_mock.go -package mock
mockgen -source ./api/metrics/prometheus_client.go -destination ./api/metrics/mock/prometheus_client_mock.go -package mock
mockgen -source ./api/environments/job_handler.go -destination ./api/environments/mock/job_handler_mock.go -package mock
mockgen -source ./api/environments/environment_handler.go -destination ./api/environments/mock/environment_handler_mock.go -package mock
mockgen -source ./api/utils/tlsvalidation/interface.go -destination ./api/utils/tlsvalidation/mock/tls_secret_validator_mock.go -package mock
Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,18 @@ The following env vars are needed. Useful default values in brackets.
- `RADIX_CONTAINER_REGISTRY` - (`radixdev.azurecr.io`)
- `PIPELINE_IMG_TAG` - (`master-latest`)
- `TEKTON_IMG_TAG` - (`release-latest`)
- `PROMETHEUS_URL` - `http://localhost:9091` use this to get Prometheus metrics running the following command (the local port 9090 is used by the API server `/metrics` endpoint, in-cluster URL is http://prometheus-operator-prometheus.monitor.svc.cluster.local:9090):
```
kubectl -n monitor port-forward svc/prometheus-operator-prometheus 9091:9090
```

You also probably want to start with the argument `--useOutClusterClient=false`. When `useOutClusterClient` is `false`, several debugging settings are enabled:
* a service principal with superpowers is used to authorize the requests, and the client's `Authorization` bearer token is ignored.
* the Radix API will connect to the currently-configured `kubectl` context and ignore `K8S_API_HOST`.
* the server CORS settings are modified to accept the `X-Requested-With` header in incoming requests. This is necessary to allow direct requests from web browser while e.g. debugging [radix-web-console](https://github.com/equinor/radix-web-console).
* verbose debugging output from CORS rule evaluation is logged to console.


If you are using VSCode, there is a convenient launch configuration in `.vscode`.

#### Validate code
Expand Down
84 changes: 83 additions & 1 deletion api/applications/applications_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"strings"

applicationModels "github.com/equinor/radix-api/api/applications/models"
"github.com/equinor/radix-api/api/metrics"
"github.com/equinor/radix-api/models"
"github.com/gorilla/mux"
)
Expand All @@ -19,17 +20,19 @@ type applicationController struct {
*models.DefaultController
hasAccessToRR
applicationHandlerFactory ApplicationHandlerFactory
prometheusHandler metrics.PrometheusHandler
}

// NewApplicationController Constructor
func NewApplicationController(hasAccessTo hasAccessToRR, applicationHandlerFactory ApplicationHandlerFactory) models.Controller {
func NewApplicationController(hasAccessTo hasAccessToRR, applicationHandlerFactory ApplicationHandlerFactory, prometheusHandler metrics.PrometheusHandler) models.Controller {
if hasAccessTo == nil {
hasAccessTo = hasAccess
}

return &applicationController{
hasAccessToRR: hasAccessTo,
applicationHandlerFactory: applicationHandlerFactory,
prometheusHandler: prometheusHandler,
}
}

Expand Down Expand Up @@ -133,6 +136,11 @@ func (ac *applicationController) GetRoutes() models.Routes {
Method: "POST",
HandlerFunc: ac.RegenerateDeployKeyHandler,
},
models.Route{
Path: appPath + "/resources",
Method: "GET",
HandlerFunc: ac.GetUsedResources,
},
}

return routes
Expand Down Expand Up @@ -992,3 +1000,77 @@ func (ac *applicationController) TriggerPipelinePromote(accounts models.Accounts

ac.JSONResponse(w, r, &jobSummary)
}

// GetUsedResources Gets used resources for the application
func (ac *applicationController) GetUsedResources(accounts models.Accounts, w http.ResponseWriter, r *http.Request) {
// swagger:operation GET /applications/{appName}/resources application getResources
// ---
// summary: Gets used resources for the application
// parameters:
// - name: appName
// in: path
// description: Name of the application
// type: string
// required: true
// - name: environment
// in: query
// description: Name of the application environment
// type: string
// required: false
// - name: component
// in: query
// description: Name of the application component in an environment
// type: string
// required: false
// - name: duration
// in: query
// description: Duration of the period, default is 30d (30 days). Example 10m, 1h, 2d, 3w, where m-minutes, h-hours, d-days, w-weeks
// type: string
// required: false
// - name: since
// in: query
// description: End time-point of the period in the past, default is now. Example 10m, 1h, 2d, 3w, where m-minutes, h-hours, d-days, w-weeks
// type: string
// required: false
// - name: ignorezero
// in: query
// description: Ignore metrics with zero value if true, default is false
// type: string
// format: boolean
// required: false
// - name: Impersonate-User
// in: header
// description: Works only with custom setup of cluster. Allow impersonation of test users (Required if Impersonate-Group is set)
// type: string
// required: false
// - name: Impersonate-Group
// in: header
// description: Works only with custom setup of cluster. Allow impersonation of a comma-seperated list of test groups (Required if Impersonate-User is set)
// type: string
// required: false
// responses:
// "200":
// description: Successful trigger pipeline
// schema:
// "$ref": "#/definitions/UsedResources"
// "404":
// description: "Not found"
appName := mux.Vars(r)["appName"]
envName := r.FormValue("environment")
componentName := r.FormValue("component")
duration := r.FormValue("duration")
since := r.FormValue("since")
ignoreZeroArg := r.FormValue("ignorezero")
var ignoreZero = false
if strings.TrimSpace(ignoreZeroArg) != "" {
ignoreZero, _ = strconv.ParseBool(ignoreZeroArg)
}

usedResources, err := ac.prometheusHandler.GetUsedResources(r.Context(), accounts.UserAccount.RadixClient, appName, envName, componentName, duration, since, ignoreZero)
if err != nil {
ac.ErrorResponse(w, r, err)
return
}

ac.JSONResponse(w, r, &usedResources)
}
Loading
Loading