Parse json into loki label

[Dubouchj]

Hello,

I couldn't find on the documentation a way to parse JSON messages into JSON logs like below:

{
"name":"John",
"age":30,
"city":"New York",
"colors":{
    "first":"red",
    "second":"blue"
   }
}

I am using Loki as an output source and use Grafana to see my logs.

Here is an example of my flows :

spec:
  filters:
  - tag_normaliser: {}
  - parser:
      key_name: message
      parse:
        type: json
      remove_key_name_field: true
      reserve_data: true
  - parser:
      key_name: color
      parse:
        type: json
  localOutputRefs:
  - loki-output

Here is the result on Grafana and I would like to parse also "color".

In the end, my goal is to add these parameters as labels instead of "detected fields".

Do you have any ideas to do it?

Best regards,
Jean-Luc

[piellick]

Hi,
There are a lot of lacks inside documentation. Fluentbit/D are not easy to use cause of officia documentation and add an abstraction layer (flow, cluster flow, output ...) does not help. No doc related to Json parsing with a fully and working example is a crime 🗡️ .

[duhow]

@Dubouchj this question is not related to logging-operator, but Loki / Grafana itself.
As long as you're getting the logs in JSON format, you will be able to query and filter them within Grafana itself.
From the configuration you shared, it should be working properly.

You can use some Loki query as:

{namespace="logging"} | json | color_first="red"

[pepov]

once data is parsed you should be able to access data (even nested) using the labels field in the loki output config:

spec:
  loki:
    url: http://loki:3100/
    labels:
      color: "$.color"

[tarokkk]

if color is structured you can use:

labels:
  color_first: $.color.first