Skip to content

Commit

Permalink
Turbo, Pagination, Server side transformation and bug fixes (#616)
Browse files Browse the repository at this point in the history
* adding turbo

* post processing

* pagination models

* pagination frontend

* bug fix for empty frame

* server side transformations. Fixes #615

* release prep

* fix linting errors

* fixes
  • Loading branch information
yesoreyeram authored Jun 28, 2023
1 parent c879e5d commit edb1978
Show file tree
Hide file tree
Showing 69 changed files with 3,614 additions and 589 deletions.
28 changes: 28 additions & 0 deletions .changeset/changelog.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
const changelogFunctions = {
getReleaseLine: async (changeset, type, options) => {
let prefix = '🎉';
if (type === 'major') {
prefix = '🎉';
} else if (type === 'minor') {
prefix = '🚀';
} else if (type === 'patch') {
prefix = '🐛';
}
if (changeset && changeset.summary) {
const summary = changeset.summary || '';
if (summary.indexOf('Docs') > -1) {
prefix = '📝';
}
if (summary.indexOf('Chore') > -1 || summary.indexOf('grafana-plugin-sdk-go') > -1 || summary.indexOf('compiled') > -1) {
prefix = '⚙️';
}
return [prefix, summary].join(' ');
}
return [prefix, changeset?.summary].join(' ');
},
getDependencyReleaseLine: async (changesets, dependenciesUpdated, options) => {
return '\n';
},
};

module.exports = changelogFunctions;
2 changes: 1 addition & 1 deletion .changeset/config.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://unpkg.com/@changesets/config@2.3.0/schema.json",
"changelog": "@changesets/cli/changelog",
"changelog": "./changelog.js",
"commit": false,
"fixed": [],
"linked": [],
Expand Down
26 changes: 12 additions & 14 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
# Change Log

## 1.5.0

### Minor Changes

- **Config editor** Updated config editor UI
- **Health Check** Added support for custom health check
- **Chore** Backend binaries are now compiled with Go 1.20.4 which contains **security patches**

### Patch Changes

- **Logging** Consuming `reportInteraction` from the **@grafana/runtime** library for reporting queries
- **Logging** Updated backend logging for improved debugging experience
- **Chore** Added changeset to the build/publish pipeline
- **docs**: updated docs about user agent header
## 2.0.0

🎉 **Pagination**: Added pagination support for json type queries ( backend parser ). Refer [this github discussion](https://github.com/yesoreyeram/grafana-infinity-datasource/discussions/601) for more details
🎉 **Transformation**: Added support for server side transformations via transformations query type. This version supports transformations such as `limit`, `filterExpression`, `computedColumn` and `summarize`. Refer [this github discussion](https://github.com/yesoreyeram/grafana-infinity-datasource/discussions/614) for more details
🚀 **Health Check**: Added support for custom health check
🚀 **Config editor**: Updated config editor UI
🐛 **Logging**: Updated backend logging for improved debugging experience
🐛 **docs**: updated docs about user agent header
🐛 **Logging**: Consuming `reportInteraction` from the `@grafana/runtime` library for reporting queries
⚙️ **Chore**: Added changeset to the build/publish pipeline
⚙️ **Chore**: update grafana backend plugin SDK from v0.159.0 to v0.162.0
⚙️ **Chore**: Backend binaries are now compiled with Go 1.20.4 which contains **security patches**

## [ 1.4.1 ]

Expand Down
5 changes: 4 additions & 1 deletion cspell.config.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
"provisioning/**",
"pkg/testsuite/**",
"**/*_test.go",
"testdata/**",
"**/testdata/**",
"src/dashboards/**",
"*SeriesProvider.ts",
"website/node_modules/**",
"website/public/**",
Expand Down Expand Up @@ -82,6 +83,8 @@
"offcanvas",
"openissues",
"openpr",
"opentelemetry",
"otel",
"outfile",
"peaceiris",
"plugincheck",
Expand Down
5 changes: 4 additions & 1 deletion docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,7 @@ services:
- GF_SECURITY_CSRF_ALWAYS_CHECK=true
- GF_ENTERPRISE_LICENSE_TEXT=$GF_ENTERPRISE_LICENSE_TEXT
# - GF_FEATURE_TOGGLES_ENABLE=trimDefaults disableEnvelopeEncryption database_metrics live-service-web-workerqueryOverLive panelTitleSearch prometheusAzureOverrideAudience publicDashboards publicDashboardsEmailSharing lokiLive featureHighlights migrationLocking storage exploreMixedDatasource newTraceViewHeader correlations cloudWatchDynamicLabels datasourceQueryMultiStatus traceToMetrics newDBLibrary validateDashboardsOnSave autoMigrateOldPanels disableAngular prometheusWideSeries canvasPanelNesting scenes disableSecretsCompatibility logRequestsInstrumentedAsUnknown dataConnectionsConsole internationalization topnav grpcServer entityStore cloudWatchCrossAccountQuerying redshiftAsyncQueryDataSupport athenaAsyncQueryDataSupport newPanelChromeUI showDashboardValidationWarnings mysqlAnsiQuotes accessControlOnCall nestedFolders accessTokenExpirationCheck showTraceId datasourceOnboarding emptyDashboardPage authnService disablePrometheusExemplarSampling alertingBacktesting editPanelCSVDragAndDrop alertingNoNormalState logsSampleInExplore logsContextDatasourceUi lokiQuerySplitting lokiQuerySplittingConfig individualCookiePreferences onlyExternalOrgRoleSync traceqlSearch prometheusMetricEncyclopedia timeSeriesTable prometheusResourceBrowserCache influxdbBackendMigration clientTokenRotation prometheusDataplane lokiMetricDataplane dataplaneFrontendFallback disableSSEDataplane alertStateHistoryLokiSecondary alertStateHistoryLokiPrimary alertStateHistoryLokiOnly unifiedRequestLog renderAuthJWT pyroscopeFlameGraph externalServiceAuth useCachingService enableElasticsearchBackendQuerying authenticationConfigUI pluginsAPIManifestKey advancedDataSourcePicker opensearchDetectVersion enableDatagridEditing
- GF_FEATURE_TOGGLES_ENABLE=publicDashboards topnav dataConnectionsConsole newPanelChromeUI emptyDashboardPage correlations nestedFolders
- GF_FEATURE_TOGGLES_ENABLE=publicDashboards topnav dataConnectionsConsole newPanelChromeUI emptyDashboardPage correlations nestedFolders advancedDataSourcePicker
- GF_PLUGIN_YESOREYERAM_INFINITY_DATASOURCE_TRACING=true
- GF_INSTANCE_OTLP_ADDRESS=${GF_INSTANCE_OTLP_ADDRESS:-foo}
- GF_INSTANCE_OTLP_PROPAGATION=${GF_INSTANCE_OTLP_PROPAGATION:-w3c}
8 changes: 4 additions & 4 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ grafana-cli --pluginUrl <ZIP_FILE_URL> plugins install yesoreyeram-infinity-data
Example:

```shell
grafana-cli --pluginUrl https://github.com/yesoreyeram/grafana-infinity-datasource/releases/download/v1.5.0/yesoreyeram-infinity-datasource-1.5.0.zip plugins install yesoreyeram-infinity-datasource
grafana-cli --pluginUrl https://github.com/yesoreyeram/grafana-infinity-datasource/releases/download/v2.0.0/yesoreyeram-infinity-datasource-2.0.0.zip plugins install yesoreyeram-infinity-datasource
```

### Install using helm chart
Expand All @@ -57,19 +57,19 @@ Example:
```yml
plugins:
- https://github.com/yesoreyeram/grafana-infinity-datasource/releases/download/v1.5.0/yesoreyeram-infinity-datasource-1.5.0.zip;yesoreyeram-infinity-datasource
- https://github.com/yesoreyeram/grafana-infinity-datasource/releases/download/v2.0.0/yesoreyeram-infinity-datasource-2.0.0.zip;yesoreyeram-infinity-datasource
```
### Install using docker
With docker, you can install the plugin using the following command. This will download the latest published version of the plugin from grafana plugins directory.
```shell
docker run -p 3000:3000 -e "GF_INSTALL_PLUGINS=yesoreyeram-infinity-datasource" grafana/grafana-enterprise:9.4.7
docker run -p 3000:3000 -e "GF_INSTALL_PLUGINS=yesoreyeram-infinity-datasource" grafana/grafana-enterprise:10.0.1
```

If you need to install a custom version of the plugin with docker, use the following command.

```shell
docker run -p 3000:3000 -e "GF_INSTALL_PLUGINS=https://github.com/yesoreyeram/grafana-infinity-datasource/releases/download/v1.5.0/yesoreyeram-infinity-datasource-1.5.0.zip;yesoreyeram-infinity-datasource" grafana/grafana-enterprise:9.4.7
docker run -p 3000:3000 -e "GF_INSTALL_PLUGINS=https://github.com/yesoreyeram/grafana-infinity-datasource/releases/download/v2.0.0/yesoreyeram-infinity-datasource-2.0.0.zip;yesoreyeram-infinity-datasource" grafana/grafana-enterprise:10.0.1
```
43 changes: 43 additions & 0 deletions docs/transformations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
slug: '/wiki/transformations'
title: 'Transformations'
previous_page_title: 'CSV'
previous_page_slug: '/wiki/csv'
next_page_title: 'Global Queries'
next_page_slug: '/wiki/global-queries'
---

## Transformations

Server side transformations are introduced in Infinity 2.0. You can perform basic data manipulation in the server side.

To perform server side transformations, you need to add a query type of **transformations**. Once this added, **this will perform server side transformation over all the previous infinity queries with backend parser**.

> **Warning**: Infinity Transformations/Server side transformations, are not available neither for non infinity datasources nor infinity queries with non-backend parsers.
## Example

![image](https://github.com/yesoreyeram/grafana-infinity-datasource/assets/153843/a3116cff-d5eb-4092-83bb-2ca17ec1f052#center)
![image](https://github.com/yesoreyeram/grafana-infinity-datasource/assets/153843/bf8b5787-e8b2-4847-95a7-544aa2f4848e#center)

## Supported transformations

### Limit

Limit transformation limits the number of result items rows in each query.

### Filter Expression

Filter the results based on column values in each query.

### Computed Column

Appends a new column based on expression

### Summarize / Group by

Group by results based on aggregation and dimension.

## Notes

> **Note**: All these transformations, are done post processing. ( once server responds with data, infinity backend client manipulates the data. ). If your API supports, such transformations on server side, use those instead of this.
38 changes: 19 additions & 19 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,15 @@ require (
github.com/google/uuid v1.3.0
github.com/gorilla/mux v1.8.0
github.com/grafana/grafana-aws-sdk v0.14.0
github.com/grafana/grafana-plugin-sdk-go v0.159.0
github.com/grafana/grafana-plugin-sdk-go v0.166.0
github.com/graphql-go/graphql v0.8.0
github.com/graphql-go/handler v0.2.3
github.com/stretchr/testify v1.8.2
github.com/xinsnake/go-http-digest-auth-client v0.6.0
github.com/yesoreyeram/grafana-framer v0.0.9
github.com/yesoreyeram/grafana-plugins/macros v0.0.0-20230116115921-668d4e86b602
go.opentelemetry.io/otel v1.14.0
go.opentelemetry.io/otel/trace v1.14.0
golang.org/x/oauth2 v0.6.0
gopkg.in/Knetic/govaluate.v3 v3.0.0
moul.io/http2curl v1.0.0
Expand All @@ -33,23 +35,23 @@ require (
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/elazarl/goproxy v0.0.0-20220115173737-adb46da277ac // indirect
github.com/fatih/color v1.7.0 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/getkin/kin-openapi v0.112.0 // indirect
github.com/go-logr/logr v1.2.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/swag v0.19.15 // indirect
github.com/go-sql-driver/mysql v1.6.0 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/flatbuffers v2.0.0+incompatible // indirect
github.com/google/go-cmp v0.5.9 // indirect
github.com/grafana/sqlds/v2 v2.3.10 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0 // indirect
github.com/hashicorp/go-hclog v0.14.1 // indirect
github.com/hashicorp/go-plugin v1.4.3 // indirect
github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d // indirect
github.com/hashicorp/go-hclog v1.5.0 // indirect
github.com/hashicorp/go-plugin v1.4.9 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/iancoleman/orderedmap v0.2.0 // indirect
github.com/invopop/yaml v0.1.0 // indirect
github.com/itchyny/gojq v0.12.7 // indirect
Expand All @@ -61,21 +63,21 @@ require (
github.com/jwalton/go-supportscolor v1.1.0 // indirect
github.com/klauspost/compress v1.15.2 // indirect
github.com/lib/pq v1.10.5 // indirect
github.com/magefile/mage v1.14.0 // indirect
github.com/magefile/mage v1.15.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattetti/filebuffer v1.0.1 // indirect
github.com/mattn/go-colorable v0.1.4 // indirect
github.com/mattn/go-isatty v0.0.14 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.18 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/mattn/go-sqlite3 v1.14.12 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/mitchellh/go-testing-interface v1.0.0 // indirect
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
github.com/noborus/tbln v0.0.1 // indirect
github.com/noborus/trdsql v0.10.0 // indirect
github.com/oklog/run v1.0.0 // indirect
github.com/oklog/run v1.1.0 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/pierrec/lz4 v2.6.1+incompatible // indirect
github.com/pierrec/lz4/v4 v4.1.8 // indirect
Expand All @@ -98,24 +100,22 @@ require (
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.40.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.37.0 // indirect
go.opentelemetry.io/contrib/propagators/jaeger v1.15.0 // indirect
go.opentelemetry.io/otel v1.14.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.14.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.14.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.14.0 // indirect
go.opentelemetry.io/otel/metric v0.37.0 // indirect
go.opentelemetry.io/otel/sdk v1.14.0 // indirect
go.opentelemetry.io/otel/trace v1.14.0 // indirect
go.opentelemetry.io/proto/otlp v0.19.0 // indirect
golang.org/x/crypto v0.0.0-20220427172511-eb4f295cb31f // indirect
golang.org/x/net v0.8.0 // indirect
golang.org/x/sys v0.6.0 // indirect
golang.org/x/term v0.6.0 // indirect
golang.org/x/text v0.8.0 // indirect
golang.org/x/net v0.9.0 // indirect
golang.org/x/sys v0.7.0 // indirect
golang.org/x/term v0.7.0 // indirect
golang.org/x/text v0.9.0 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230110181048-76db0878b65f // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
google.golang.org/grpc v1.54.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
google.golang.org/protobuf v1.30.0 // indirect
gopkg.in/fsnotify/fsnotify.v1 v1.4.7 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Loading

0 comments on commit edb1978

Please sign in to comment.