v1.0.b6 (#73)

* update of readme.md * feature implemented for buddy boards (#52) * Fix/einsy (#53) * update config.go and workaround for einsy boards connection test * fixed scraping of einsy * updated to support printer autodetection * metrics update * improved api access * updated einsy speed to be like buddy metric * updated config.go * Experimental/syslog (#54) * update config for exporter * primitive syslog server is up and running * added switch to config * defined list of metrics * added samples of data that syslog produces * added readme for syslog * very first primitive version of parsing * tidy and pattern update * update of patterns * added mac address * upload of parsed data example * moved parsed sample to markdown * removed debug loop * update of list of parsed metrics * updated patterns and expanded list of metrics * updated and optimized syslog logic * added warning for syslog metrics * defined few prometheus metrics for syslog and updated list of scraped data * defined new prometheus metrics * described syslog prometheus metrics * first version of prometheus Collect function done * fixed name of keys in collector * update of cpu metric to be ratio * update of documentation to acknowledge syslog * dashboard update * slight adjustment of dashboard * updated image in readme * fixed error message * fixed disabling syslog * reformating readme.md * update roadmap * Feature/dashboards (#55) * updated units for syslog metrics * buddy dashboard - change time interval - added nozzle diameter panel * update of buddy image * updated Einsy dashboard * updated readme * updated panel with temperatures - added units * updated overview dashboad * updated readme * updated einsy dashboard to support status flags * updated buddy.png * removed old legacy dashboards * Fix/metrics (#56) * fixed nozzle temperature for XL * Einsy status fix * Fix/misc (#57) * fixed logs at startup * various fixes of README.md * fixed issue with duplicate prusa.yml * moved prusa.yml * missing items in roadmap * fixed missing config.gcode * updated docker-compose.yaml * removed old on-premise yaml docker compose * removal of deprecated folder * removed deprecated metrics from einsy * fixed config reloading * bump versions for rpi image * audit changes * bump golang version * fixed golang version in docker.yml * updated go.mod * updated syslog buddy image in README.md * roadmap update * update of roadmap * EXPERIMENTAL Features/pritner logs (#58) * implemented logging service for printers - push to loki like server * added loki_push_api to promtail * updated docker-compose.yaml * updated README.md * update of README.md * prebuild commit * added forgotten files * updated for logs * Experimental/syslog update (#59) * fix main.go * updated samples of syslog data * updated config.gcode with more metrics * metrics_example.md update * updated list of syslog metrics * updated logging - should not panic now - moved logging service to syslogLoki.go * changed port to whole address with port * refactored syslog code * rename collectLogs to handleLogs and collectMetrics to handleMetrics * update of list of metrics in gcode * added new syslog metrics * Features/sl support (#60) * implemented access api for SL printers * added code for scraping sl printers * added api responses from sl printer * sl partially supported * status metric fix * updated readme.md * overview dashboard support for sl1 * added dashboard for resin printers * update of config * update of README.md * fix image of sl dashboard * bump version of exporter in docker compose * Docs/api (#61) * added buddy api answers * added einsy api answers * added sl api answers * created prusa_link folder and moved responses there * sl added list of api * buddy added list of api endpoints * Refactoring (#62) * moved prusa link related .go files into prusa_link * moved syslog.go into syslog/syslog.go * created exporter related folder * moved files here and there * config.go package rename * main.go package rename * config package * renaming prusa_link to prusalink * renamed package * Revert "Refactoring (#62)" (#64) This reverts commit 2d1c08d. * Refactoring (#69) * moved prusa link related .go files into prusa_link * moved syslog.go into syslog/syslog.go * created exporter related folder * moved files here and there * config.go package rename * main.go package rename * config package * renaming prusa_link to prusalink * renamed package * reverting * created structs for Prusa Link * created .go files for buddy, einsy and sl * Update package name in prusalink files * created new syslog.go * created file what should I need to do while sending metrics enabler to printer * updated config.gcode with new metrics * created common.go * updating configuration * created new main.go * updated config.go - finalized * addded func for log leve * updated main.go to contain cmd.Run() * moving files to depr folder and marking them as deprecated * removing junk * kingpin/v2 implemenation * created function accessPrinterEndpoint * returning prusa.yml to deprecated folder * error handling request with api key * updated flag getter * created few maps that will help us later * updated accessPrinterEndpoint function * get functions * GetFiles is recursive now * ProbePrinter function * probing config file * updated prober * metrics definition for syslog * scraping buddy * scraping sl * scraping einsy * syslog collector * enabling syslog collector * config reloader * buddy collector * einsy collector * SL collector * buddy metrics exposing * syslog metrics exposing * sl and einsy metrics exposing * removed duplicate configuration declaration * registered metrics * removed duplication of configuration in prusalink * updated scrapers * split syslog go files * started merging prometheus prusalink metrics * refactoring metrics * changed collector * moving scrape functions * cleaning prusalink folder * renamed common.go to prusalink.go * renaming prusa_buddy_... syslog metrics to prusa_... only same as in prusalink and basically merging them * reformating syslog metrics * dropping syslog prefix from prometheus functions in syslog/prometheus.go * moving config reference for prusalink * updated metrics Collect function for prusalink to contain metrics for einsy buddy and sl * metrics updated for prusalink buddy * separate common metrics to standalone function * Remove unnecessary print statements * Fix error handling in probeConfigFile function * updated structs to float64 * Remove unused 'sd' field and change 'Files' struct fields to float64 * Fixed probing of einsy based printers * Fix log level and handle error in configuration probing * Remove unused fields from Files struct * Refactor error logging in getCommonMetrics function * Removed standalone scraping functions * Removed printerZHeight metric - replaced with prusa_axis_z * Refactor Prometheus metrics collection and remove unused code * Fixed sl metrics and little bit splitted getCommonMetrics function * Moved einsy metrics block and updated error logging * Refactor printer scraping code and add timeout to HTTP client * Reorganised logic of prusalink metrics * Raw metrics from syslog kinda prepared for Prometheus * slight upgrade of syslog metrics * Fixed prusa_current_mmu_imp to prusa_current_mmu_inp - typo in firmware * Merged PrusaLink prusa_fan metrics to prusa_fan_speed * Merged prusa_axis_ metrics into one * Updated syslog metrics - let's expose them * Syslog metrics update * Alphabetically sorted collector * Alphabetically sorted NewCollector * Describe sorted as well * Removed Buddy from BuddySyslogInfo... * Updated config.gcode for syslog. In total 134 metrics. Let it burn * updated patterns and syslog parsing * Migrated syslogMetrics to sync.Map{} * Commented several parts of code * Removed label struct that was not used * Prototyped new syslog metrics * Dynamic metrics expose prototype * Even more separating syslog and prusalink * Attempting for fully dynamic exposing * Removing getField function * SYSLOG Metrics (#70) * Logging update for syslog/prometheus.go * Removed commented code from syslog/prometheus.go * Moving metric descriptions to map * Moved metrics definition to standalone golang file * Fixed typo * Overriding for few syslog metrics * Reverting few changes * Sylog metrics updated * Updated switching between metrics * Fixed few metrics and fixed issue with func getNumberOf * Exposing metrics updated * Removed metrics.go * Created collector.go * Fixed few metrics and dropped label port * Added TTL so I can detect that printer is down * config update * Updated config - PrusaLink metrics can be now disabled * Updated README.md * Fixed typo in Prusalink dashboard * Removed deprecations * Updated PrusaLink dashboard * Hotfix PrusaLink dashboard * Updated Prusa Overview dashboard * Removed deprecated dashboards * Mock of syslog dashboard * Temporarily disabling setting Rachable state to false while scraping the printer * Temporarily diabling Reachable state while probing * Temporarily disabling checking if printer is reachable * Fixed cpu_usage metric * Updated syslog dashboard and uploaded to Grafana.net * Updated README.md * Created logs.go for Syslog * Configuration update to support Logs * Updated logging functions - logs into file now with log rotation * Simplified logs and unhardcoding variables * Updated handling error while creating directory for printer logs * Agent configuration update * Docker compose for testing * Updated agent.yaml * Config gcodes updated * Changing directory for printer logs * Bump version to beta 5 * Updated rpi.yml pipeline * Updated prusa_current for other than dwarfs. Those were divided by 1000 in firmware * Removing garbage * Scraped config reloader * Paralelized PrusaLink scraping Remove unused code and print statements * Hotfixing prusa_material * Moved WaitGroup addition inside of the for loop for PrusaLink * Unhardcoded Timeout for PrusaLink * README.md update * README.md overhaul * Fixed rpi_image.md * Fixed syslog.md * Hotfixed syslog.md * Temporarily adding script for building binaries * Updated time format for logs to UnixNs Updated promtail parsing of logs from printer Updated printer logs to supports all labels * Updated Grafana agent example config * Updated version of exporter in docker compose
pstrobl96 · Mar 6, 2024 · 896ae84 · 896ae84
1 parent 6037504
commit 896ae84
Show file tree

Hide file tree

Showing 21 changed files with 1,157 additions and 1,026 deletions.
diff --git a/.github/workflows/rpi.yml b/.github/workflows/rpi.yml
@@ -39,14 +39,14 @@ jobs:
           sh ./get-docker.sh
           systemctl enable docker
 
-          curl https://github.com/pstrobl96/prusa_exporter/blob/releases/v1.0.0/docs/examples/config/common/prusa.yml -o /boot/firmware/prusa.yml
+          curl https://raw.githubusercontent.com/pstrobl96/prusa_exporter/main/docs/examples/config/grafana_cloud/prusa.yml -o /boot/firmware/prusa.yml
           curl https://raw.githubusercontent.com/pstrobl96/prusa_exporter/main/docs/examples/config/grafana_cloud/agent.yaml -o /boot/firmware/agent.yaml
 
           rm -rf /opt/prusa_exporter
 
           mkdir /opt/prusa_exporter
-          curl https://github.com/pstrobl96/prusa_exporter/blob/releases/v1.0.0/docker-compose.rpi-cloud.yaml -o /opt/prusa_exporter/docker-compose.yaml
-
+          curl https://raw.githubusercontent.com/pstrobl96/prusa_exporter/main/docker-compose.rpi-cloud.yaml -o /opt/prusa_exporter/docker-compose.yaml
+          
           curl https://raw.githubusercontent.com/pstrobl96/prusa_exporter/main/docs/services/prusa-exporter-docker.service -o /etc/systemd/system/prusa-exporter.service
           systemctl enable prusa-exporter
 

diff --git a/README.md b/README.md
diff --git a/build.sh b/build.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+
+platforms=("linux/amd64" "linux/arm" "linux/riscv64" "linux/arm64" "windows/amd64" "darwin/amd64" "windows/arm64" "darwin/arm64")
+
+output_dir="bin"
+
+mkdir -p $output_dir
+
+for platform in "${platforms[@]}"
+do
+    GOOS=$(echo $platform | cut -d'/' -f1)
+    GOARCH=$(echo $platform | cut -d'/' -f2)
+
+    output_name="$output_dir/prusa_exporter-$GOOS-$GOARCH"
+    if [ $GOOS = "windows" ]; then
+        output_name="$output_name.exe"
+    fi
+
+    echo "Building $output_name ..."    
+    env GOOS=$GOOS GOARCH=$GOARCH go build -o $output_name
+done
+
+echo "Build completed."
diff --git a/cmd/main.go b/cmd/main.go
@@ -1,11 +1,9 @@
 package cmd
 
 import (
-	"fmt"
 	"net/http"
 	"os"
 	"strconv"
-	"time"
 
 	"github.com/alecthomas/kingpin/v2"
 	"github.com/prometheus/client_golang/prometheus"
@@ -25,8 +23,6 @@ var (
 	metricsPort      = kingpin.Flag("exporter.metrics-port", "Port where to expose metrics.").Default("10009").Int()
 	syslogTTL        = kingpin.Flag("syslog.ttl", "TTL for syslog metrics in seconds.").Default("60").Int()
 	prusalinkTimeout = kingpin.Flag("prusalink.timeout", "Timeout for prusalink requests in ms.").Default("1000").Int()
-	// Configuration used for scraping and exporter
-	Configuration config.Config
 )
 
 // Run function to start the exporter
@@ -47,6 +43,7 @@ func Run() {
 		logLevel = zerolog.InfoLevel // default log level
 	}
 	zerolog.SetGlobalLevel(logLevel)
+	zerolog.TimeFieldFormat = zerolog.TimeFormatUnixNano
 
 	config, err = probeConfigFile(config)
 
@@ -58,11 +55,7 @@ func Run() {
 
 	if config.Exporter.Prusalink.Enabled {
 		log.Info().Msg("PrusaLink metrics enabled!")
-		collectors = append(collectors, prusalink.NewCollector(&config))
-	}
-
-	if config.Exporter.ReloadInterval != 0 { // do not run reloader if interval is set to zero
-		go configReloader(&config, *configReload) // run reloader as goroutine
+		collectors = append(collectors, prusalink.NewCollector(config))
 	}
 
 	if config.Exporter.Syslog.Metrics.Enabled {
@@ -102,33 +95,16 @@ func probeConfigFile(config config.Config) (config.Config, error) {
 
 		if err != nil {
 			log.Error().Msg(err.Error())
-			//printer.Reachable = false
+			printer.Reachable = false
 		} else if status {
 			printerType, err := prusalink.GetPrinterType(printer)
 			if err != nil || printerType == "" {
 				log.Error().Msg(err.Error())
 				printer.Type = "unknown"
 			}
 			config.Printers[i].Type = printerType
-			//config.Printers[i].Reachable = status
+			config.Printers[i].Reachable = status
 		}
 	}
 	return config, nil
 }
-
-func configReloader(configuration *config.Config, reloadInterval int) {
-	ticker := time.NewTicker(time.Duration(reloadInterval) * time.Second)
-
-	for t := range ticker.C {
-		log.Info().Msg(fmt.Sprintf("Config reloaded at: %v\n", t.UTC()))
-		config, err := config.LoadConfig(*configFile)
-		if err != nil {
-			log.Error().Msg("Error loading configuration file " + err.Error())
-		}
-		config, err = probeConfigFile(config)
-		if err != nil {
-			log.Error().Msg("Error probing configuration file " + err.Error())
-		}
-		configuration = &config
-	}
-}
diff --git a/config/config.go b/config/config.go
@@ -10,10 +10,9 @@ import (
 // Config struct for the configuration file prusa.yml
 type Config struct {
 	Exporter struct {
-		ScrapeTimeout  int    `yaml:"scrape_timeout"`
-		ReloadInterval int    `yaml:"reload_interval"`
-		LogLevel       string `yaml:"log_level"`
-		Prusalink      struct {
+		ScrapeTimeout int    `yaml:"scrape_timeout"`
+		LogLevel      string `yaml:"log_level"`
+		Prusalink     struct {
 			Enabled bool `yaml:"enabled"`
 		} `yaml:"prusalink"`
 		Syslog struct {

diff --git a/docker-compose.rpi-cloud.yaml b/docker-compose.rpi-cloud.yaml
@@ -8,7 +8,7 @@ networks:
 
 services:
   exporter:
-    image: pubeldev/prusa-exporter:v1.0.b5
+    image: pubeldev/prusa-exporter:v1.0.b6
     container_name: exporter
     restart: unless-stopped
     volumes:
@@ -29,6 +29,7 @@ services:
     container_name: grafana_agent
     restart: unless-stopped
     volumes:  
+      - /var/run/docker.sock:/var/run/docker.sock
       - prusa_syslog:/var/log/prusa
       - type: bind
         source: /boot/firmware/agent.yaml

diff --git a/docker-compose.testing.yaml b/docker-compose.testing.yaml
@@ -29,6 +29,7 @@ services:
     container_name: grafana_agent
     restart: unless-stopped
     volumes:  
+      - /var/run/docker.sock:/var/run/docker.sock
       - prusa_syslog:/var/log/prusa
       - type: bind
         source: ./agent.yaml

diff --git a/docs/config.md b/docs/config.md
@@ -0,0 +1,90 @@
+# Configuration of Grafana Agent / Prometheus / Promtail / Loki
+
+## agent.yml
+
+Grafana Agent is used in Raspberry Pi image and currently works only with Grafana Cloud - if you don't configure it different way. You need to change `url`, `username` and `password`. You can get these values in configuration of your Grafana Cloud. How you can find in [Grafana Cloud documentation](https://grafana.com/docs/grafana-cloud/send-data/metrics/metrics-prometheus/). Example configuration can be found in [agent.yaml](examples/config/grafana_cloud/agent.yaml)
+
+```
+metrics:
+  global:
+    scrape_interval: 15s
+    remote_write:
+    - url: <YOUR CLOUD METRICS URL>
+      basic_auth:
+        username: "<YOUR CLOUD METRICS USERNAME>"
+        password: "<YOUR CLOUD METRICS PASSWORD>"
+```
+
+If you want to parse logs from printer you can use this pipeline
+
+```
+logs:
+  positions_directory: /var/lib/grafana-agent
+  configs:
+  - name: prusa
+    clients:
+    - url: <url>
+    scrape_configs:
+    - job_name: system
+      pipeline_stages:
+      static_configs:
+      - labels:
+          job: varlogs
+          __path__: /var/log/*.log
+    - job_name: prusa_syslog
+      pipeline_stages:
+        - json:
+            expressions:
+              message:
+              stream:
+              time:
+              level:
+              app_name:
+              client:
+              hostname:
+        - labels:
+            stream:
+            time:
+            level:
+            app_name:
+            client:
+            hostname:
+        - output:
+            source: message
+      static_configs:
+      - labels:
+          stream: 'stdout'
+          job: prusa_printers_logs
+          __path__: /var/log/prusa/exporter.log
+```
+
+## prometheus.yml
+
+In [prometheus.yml](examples/config/on_premise/prometheus.yml) you need to change the `remote_write` section. This section is responsible for writing data to Grafana Cloud instance. You can get all values in config of your Grafana instance. You can get more information in [Grafana Docs](https://grafana.com/docs/grafana-cloud/data-configuration/metrics/metrics-prometheus/).
+
+| key      | value                                  |
+|----------|----------------------------------------|
+| url      | this is where your instance is running |
+| username | name that is used for login            |
+| password | unique key used for login              |
+
+```
+remote_write:
+- url: https://prometheus-prod-01-eu-west-0.grafana.net/api/prom/push
+  basic_auth:
+    username: userName
+    password: apiKey
+```
+
+## promtail.yml
+
+In [promtail.yml](examples/config/on_premise/promtail.yml) you need to change the `clients` section. Thanks to this block promtail will send logs to your Grafana Cloud Loki instance instead of local Loki. More details of log ingestion in [Grafana docs](https://grafana.com/docs/grafana-cloud/data-configuration/logs/collect-logs-with-promtail/).
+
+| key      | value                                                 |
+|----------|-------------------------------------------------------|
+| url      | this is string that you can generate in Grafana Cloud |
+
+```
+clients:
+  - url: https://<User Name>:<Your Grafana.com API Key>@logs-prod-eu-west-0.grafana.net/loki/api/v1/push
+```
diff --git a/docs/examples/config/common/prusa.yml b/docs/examples/config/common/prusa.yml
@@ -1,6 +1,5 @@
 exporter:
-  scrape_timeout: 1 # scrape timeout of Prusa Link
-  reload_interval: 300 # interval in seconds for config reloader
+  scrape_timeout: 1000 # scrape timeout of Prusa Link in ms
   log_level: info
   prusalink:
     enabled: true

diff --git a/docs/examples/config/grafana_cloud/agent.yaml b/docs/examples/config/grafana_cloud/agent.yaml
@@ -23,32 +23,78 @@ logs:
     clients:
     - url: <url>
     scrape_configs:
+    - job_name: docker
+      pipeline_stages:
+        - json:
+            expressions:
+              message:
+              stream: 
+              time:   
+              level: 
+        - labels:
+            stream:
+            time:   
+            level: 
+        - output:
+            source: message
+        - timestamp:
+            source: time
+            format: UnixNs
+      docker_sd_configs:
+        - host: unix:///var/run/docker.sock
+          refresh_interval: 5s
+      relabel_configs:
+        - source_labels: ['__meta_docker_container_name']
+          regex: '/(.*)'
+          target_label: 'container'
+        - source_labels: ['__meta_docker_container_log_stream']
+          target_label: 'logstream'
+        - source_labels: ['__meta_docker_container_label_logging_jobname']
+          target_label: 'job'
     - job_name: system
       pipeline_stages:
       static_configs:
       - labels:
           job: varlogs
-          __path__: /var/log/*.log
+          __path__: /var/log/*.log 
     - job_name: prusa_syslog
       pipeline_stages:
         - json:
             expressions:
               message:
-              stream:
-              time:
-              level:
-              app_name:
-              client:
-              hostname:
+              stream: 
+              time:   
+              level: 
+              app_name: 
+              client: 
+              hostname: 
+              priority:
+              proc_id:
+              msg_id:
+              severity:
+              facility:
+              structured_data:
+              tls_peer:
+              version:
         - labels:
             stream:
-            time:
-            level:
-            app_name:
-            client:
-            hostname:
+            level: 
+            app_name: 
+            client: 
+            hostname: 
+            priority:
+            proc_id:
+            msg_id:
+            severity:
+            facility:
+            structured_data:
+            tls_peer:
+            version:
         - output:
             source: message
+        - timestamp:
+            source: time
+            format: UnixNs
       static_configs:
       - labels:
           stream: 'stdout'