Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filterx parse csv #116

Merged
merged 6 commits into from
May 23, 2024
Merged

Filterx parse csv #116

merged 6 commits into from
May 23, 2024

Conversation

bshifter
Copy link
Member

CSVParser functionality ported to filterx:

filterx-func-parse-csv function added with the support of:

  • custom column names
  • delimiters
  • dialects
  • flag: greedy
  • flag: strip_whitespace

It uses the new 'named function arguments' to handle options, so all the option arguments are optional. The only positional argument supported by the function is the target string to be parsed.

known issues:
filterx function framework does not support inline json literals, this way the custom columns argument need to be built in a separate command (see below). however, the interpretation of column's values can not be made on preprocess, so the columns' expression evaluation happens at run time (on every _eval). This will be fixed in the future, since this workaround's performance cost is relative high.

example usage:

filterx { cols = json_array(["foo", "bar", "baz"]); res = parse_csv($MSG, columns=cols, delimiters=",;:", dialect="escape-backslash-with-sequences", greedy=true, strip_whitespace=false); }

bshifter added 2 commits May 22, 2024 17:22
Signed-off-by: shifter <shifter@axoflow.com>
Signed-off-by: shifter <shifter@axoflow.com>
@bshifter bshifter force-pushed the filterx-parse-csv branch from 808f52b to 3ed8ee9 Compare May 22, 2024 15:22
bshifter added 4 commits May 22, 2024 18:41
…d optional named args in filterx function

Signed-off-by: shifter <shifter@axoflow.com>
Signed-off-by: shifter <shifter@axoflow.com>
Signed-off-by: shifter <shifter@axoflow.com>
Signed-off-by: shifter <shifter@axoflow.com>
@bshifter bshifter force-pushed the filterx-parse-csv branch from 3ed8ee9 to 7f7ab48 Compare May 22, 2024 16:42
@bshifter bshifter requested a review from alltilla May 22, 2024 17:24
@alltilla alltilla merged commit b7d4414 into axoflow:main May 23, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants