filterx: load_vars()

[alltilla]

Inverse of vars().

Note: FilterX level variables are loaded and declared.

[OverOrion]

Overall LGTM, I only have scenario that I think is worth a short discussion:
What should happen if the to be loaded json contains a $MSG as key for example?
The current behavior is that the json's $MSG key is resolved first (e.g., "foo") and then it will be loaded as {"foo": "some_val"}.
I am not certain whether the user would have wanted to actually overwrite $MSG or not in this case 😕.

log {
  source{stdin(flags(no-parse));};

  filterx{
    my_vars = {"var1":"foo","var2":["foo","bar"],"$MSG":"msg_overwritten"};

    load_vars(my_vars);

    vars();
  };

  destination{stdout();};
};

[2024-11-26T13:07:37.132156] Incoming log entry; input='hello', msg='0x7955a8011cc0', rcptid='5644'
....
[2024-11-26T13:07:37.132156] FILTERX ESTEP; expr='foobar.conf:13:5|     vars()', value='{"$MESSAGE":"hello","my_vars":{"var1":"foo","var2":["foo","bar"],"hello":"msg_overwritten"},"var1":"foo","var2":["foo","bar"],"hello":"msg_overwritten"}', truthy='1', type='json_object'
....
[2024-11-26T13:07:37.133140] Outgoing message; message='Nov 26 13:07:37 orion-T14G3 hello\x0a'

[alltilla]

@OverOrion

That is caused by an unintuitive template resolution, I think.
"$MESSAGE" gets resolved, but in this case I don't think you want it resolved.
If you workaround the template resolution strangeness, like in the light test, or you load the JSON from outside, it works as expected.

syslog-ng.conf:

log {
  source { file("/path/to/vars.json" flags(no-parse)); };

  filterx {
    load_vars(json($MESSAGE));
  };

  destination { stdout(template("$MESSAGE\n")); };
};

vars.json:

{"$MESSAGE":"foo","pipeline_level_variable":"baz","log":{"body":"foobar","attributes":{"attribute":42}},"js_array":[1,2,3,[4,5,6]]}