Google Stackdriver Prometheus Exporter

A Prometheus exporter for Google Stackdriver Monitoring metrics. It acts as a proxy that requests Stackdriver API for the metric's time-series everytime prometheus scrapes it.

Installation

Binaries

Download the already existing binaries for your platform:

$ ./stackdriver_exporter <flags>

From source

Using the standard go install (you must have Go already installed in your local machine):

$ go install github.com/frodenas/stackdriver_exporter
$ stackdriver_exporter <flags>

Docker

To run the stackdriver exporter as a Docker container, run:

$ docker run -p 9255:9255 frodenas/stackdriver-exporter <flags>

Cloud Foundry

The exporter can be deployed to an already existing Cloud Foundry environment:

$ git clone https://github.com/frodenas/stackdriver_exporter.git
$ cd stackdriver_exporter

Modify the included application manifest file to include the desired properties. Then you can push the exporter to your Cloud Foundry environment:

$ cf push

BOSH

This exporter can be deployed using the Prometheus BOSH Release.

Usage

Credentials and Permissions

The Google Stackdriver Exporter uses the Google Golang Client Library, which offers a variety of ways to provide credentials. Please refer to the Google Application Default Credentials documentation to see how the credentials can be provided.

If you are using IAM roles, the monitoring.metricDescriptors.list and monitoring.timeSeries.list IAM permissions are required. The roles/monitoring.viewer IAM role contains those permissions. See the Access Control Guide for more information.

If you are still using the legacy Access scopes, the https://www.googleapis.com/auth/monitoring.read scope is required.

Flags

Flag / Environment Variable	Required	Default	Description
google.project-id STACKDRIVER_EXPORTER_GOOGLE_PROJECT_ID	Yes		Google Project ID
monitoring.metrics-type-prefixes STACKDRIVER_EXPORTER_MONITORING_METRICS_TYPE_PREFIXES	Yes		Comma separated Google Stackdriver Monitoring Metric Type prefixes (see example and available metrics)
monitoring.metrics-interval STACKDRIVER_EXPORTER_MONITORING_METRICS_INTERVAL	No	5m	Metric's timestamp interval to request from the Google Stackdriver Monitoring Metrics API. Only the most recent data point is used
monitoring.metrics-offset STACKDRIVER_EXPORTER_MONITORING_METRICS_OFFSET	No	0s	Offset (into the past) for the metric's timestamp interval to request from the Google Stackdriver Monitoring Metrics API, to handle latency in published metrics
web.listen-address STACKDRIVER_EXPORTER_WEB_LISTEN_ADDRESS	No	:9255	Address to listen on for web interface and telemetry
web.telemetry-path STACKDRIVER_EXPORTER_WEB_TELEMETRY_PATH	No	/metrics	Path under which to expose Prometheus metrics

Metrics

The exporter returns the following metrics:

Metric	Description	Labels
stackdriver_monitoring_api_calls_total	Total number of Google Stackdriver Monitoring API calls made	project_id
stackdriver_monitoring_scrapes_total	Total number of Google Stackdriver Monitoring metrics scrapes	project_id
stackdriver_monitoring_scrape_errors_total	Total number of Google Stackdriver Monitoring metrics scrape errors	project_id
stackdriver_monitoring_last_scrape_error	Whether the last metrics scrape from Google Stackdriver Monitoring resulted in an error (1 for error, 0 for success)	project_id
stackdriver_monitoring_last_scrape_timestamp	Number of seconds since 1970 since last metrics scrape from Google Stackdriver Monitoring	project_id
stackdriver_monitoring_last_scrape_duration_seconds	Duration of the last metrics scrape from Google Stackdriver Monitoring	project_id

Metrics gathered from Google Stackdriver Monitoring are converted to Prometheus metrics:

• Metric's names are normalized according to the Prometheus specification using the following pattern:
  1. namespace is a constant prefix (stackdriver)
  2. subsystem is the normalized monitored resource type (ie gce_instance)
  3. name is the normalized metric type (ie compute_googleapis_com_instance_cpu_usage_time)
• Labels attached to each metric are an aggregation of:
  1. the unit in which the metric value is reported
  2. the metric type labels (see Metrics List)
  3. the monitored resource labels (see Monitored Resource Types)
• For each timeseries, only the most recent data point is exported.
• Stackdriver GAUGE and DELTA metric kinds are reported as Prometheus Gauge metrics; Stackdriver CUMULATIVE metric kinds are reported as Prometheus Counter metrics.
• Only BOOL, INT64, DOUBLE and DISTRIBUTION metric types are supported, other types (STRING and MONEY) are discarded.
• DISTRIBUTION metric type is reported as a Prometheus Histogram, except the _sum time series is not supported.

Example

If we want to get all CPU (compute.googleapis.com/instance/cpu) and Disk (compute.googleapis.com/instance/disk) metrics for all Google Compute Engine instances, we can run the exporter with the following options:

stackdriver_exporter \
  --google.project-id my-test-project \
  --monitoring.metrics-type-prefixes "compute.googleapis.com/instance/cpu,compute.googleapis.com/instance/disk"

Contributing

Refer to the contributing guidelines.

License

Apache License 2.0, see LICENSE.