prusa_exporter future

[pstrobl96]

The issue with the prusa_exporter is it's not necessary a complexity. Wanting to be able handle everything went into somewhat incomprehensible code. This seconds issue with wrong approach to processing syslog metrics that could be done dynamically instead of statically. This is the reason I choose to write this proposal that will be used as a base for new approach to exporter and not only that.

Split

First thing I'll implement is splitting the exporter into mutliple parts. Even though prusa_exporter was at first build for exposing Prusa Link metrics at this moment mostly handles syslog metrics. With that in mind I'll ditch Prusa Link metrics from prusa_exporter and move them into separate project named prusa_link_exporter. Second functionality I want to extract is logs processing capability, very crucial part that will be in separate project named prusa_log_processor

prusa_exporter

New purpose of prusa_exporter is going to be only to process syslog metrics that are "exposed" via line_prototcol printer sends to Influx. Issue is that proper line_protocol have whole timestamp per metric but Prusa printers does not have enough processing power and instead it uses only delta timestamp. This is the issue why some kind of man-in-the-middle is needed. In this case prusa_exporter.

I want to expose metrics at /metrics endpoint obviously but I want also use remote_write to send metrics straight into Prometheus. Also processing would be dynamic while mapping of names of metrics is going to be saved in config file. If no config is found exporter will simply use name of metric that printer sends. This allowss dynamic processing even new metrics that could be implemented in the future.

prusa_link_exporter

I want to tame json_exporter because Prusa Link is nothing else than REST API so use of json_exporter makes sense to me. Issue is going to be that there is different versions of Prusa Link but after initial setup it should be pretty easy maintain the project.

prusa_log_processor

This part of the trio would be responisble for log processing Specifically rewriting RFC5424 format into something that will Promtail be able to process. I have to try Grafana Alloy. If it already works, Promtail is unable to process logs without line break. - grafana/loki#12436

[gouthamve]

prusa_link_exporter
I want to tame json_exporter because Prusa Link is nothing else than REST API so use of json_exporter makes sense to me. Issue is going to be that there is different versions of Prusa Link but after initial setup it should be pretty easy maintain the project.

Interesting direction. I think there is a low burden way to maintain the prusa_link setup tbh that is different from json_exporter, I'll maybe open a PR in a week or so and lets evaluate that approach as well?

But in general, it looks like a good approach.

[pstrobl96]

At first prusa_link_exporter would be only extracted from prusa_exporter. I would like to migrate to json_exporter but this decision can change in the future. Main focus is at "syslog" logs and metrics at this moment. I'm working on logs part so we can evaluate prusa_exporter later after you open PR. 🎉

[RichiH]

There's two thoughts here: One the one hand, leveraging more standard components is good. E.g. use https://github.com/prometheus/exporter-toolkit or so for plain metrics pipeline.

For logs parsing, I am torn between "all in one" locally -- and simply putting it all into Loki and going from there.

On the other hand: One thing that keeps making Prometheus, Grafana, etc successful is the installation and operational simplicity. Having one component to install as the end user is better than having ten different ones. I don't know how much could be stolen from Alloy here, but I think it's worth at least considering.

[pstrobl96]

I plan to create also repository with example setups - something like prusa_observability_collector maybe? It would be bunch of docker-composes for Grafana Cloud, local setups etc.

The issue I have is that I'm kinda unable to update the code within this repository. So splitting is mainly for my own sanity.

prusa_log_processor may be unnecessary, haven't studied Alloy yet but if it works same as Promtail, then I cannot process logs without without some component between printer and Loki. Apart of thet, there is no need for it... Well you can synchronize metrics with logs. But then metric and log part have to be merged again.

For me it would be best to integrate Prometheus metrics into firmware. Maybe I could use remote_write but printers does not have enough computational power - like 640kB of memory is enough for every amirite? - to expose /metrics endpoint. But yeah, maybe.

[pstrobl96]

Ok, I tested Grafana Alloy as well and it also returns this error

alloy    | ts=2024-08-14T15:45:45.920640161Z level=warn msg="error parsing syslog stream" component_path=/ component_id=loki.source.syslog.logs_prusa_syslog err="unexpected EOF"

[RichiH]

My point re Alloy was that it combines many distinct binaries into one binary -- the code is separate, but the end user has one binary to deal with