Consider a redesign of the mutelist tags feature

[dlouzan]

New feature motivation

This is a bit of a follow-up / feedback of the discussions in a previous bug report #4782 around mutelists and tags behaviour.

After having implemented mutelists for 3 different AWS accounts in the last weeks, I have realized that when working with tag combinations, the syntax gets really messy and leads to easy errors.

The main problem stems from the behaviour of tags: they are ANDed, which is fine for some use cases, but for others, the only option to OR them is to create complex regex rules inside single yaml array elements.

As an example, this is the syntax I ended up using for the following use case: Filter out any failures on resources tagged with Name a.domain.com OR b.domain.com OR c.domain.com:

        "some_prowler_check":
          Regions:
            - "*"
          Resources:
            - "*"
          Tags:
            - "Name=(\
                a\\.domain\\.com|\
                b\\.domain\\.com|\
                c\\.domain\\.com\
              )"

The syntax above is combined with yaml flow scalar style ending each line with \ to make it a bit more readable in multiple lines, otherwise we have to write the regex in a single line, which gets unwieldly really fast. The need to make ORs in the regex via | also makes it quite error-prone.

If we have to combine tags it gets even worse, e.g. Mute for any host that has as Name any of the values as the example above, OR Type someType1 OR someType2:

        "some_prowler_check":
          Regions:
            - "*"
          Resources:
            - "*"
          Tags:
            - "Name=(\
                a\\.domain\\.com|\
                b\\.domain\\.com|\
                c\\.domain\\.com\
              )|\
              Type=(\
                someType1|\
                someType2\
              )"

All of the complications of the syntax are compounded by having to escape regex-interpreted characters inside yaml with double quotes, making the code quite unreadable. Plus regexes are not anchored, so we run also the danger of matching parts of the strings unintendedly.

Solution Proposed

I think a new syntax for being able to combine AND/OR on mutelist tag rules should be reviewed. Probably with a default behaviour that keeps current behaviour so current deployments are not broken, but that allows to overwrite the behaviour as required.

Inspiration could e.g. be taken from filebeat's processors syntax for and/or in yaml: https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html. There's no need I think for something as complicated as their support for different filters, but the and/or/not functionality would be very welcome.

Describe alternatives you've considered

Of course, if you have a better approach for implementing such a use case, I'll be happy to learn something 👍

Additional context

Thank you very much for the great project!

/cc @pedrooot

[sergargar]

Hi @dlouzan, what would your ideal syntaxes for the above examples?

In Prowler, you can also use the following syntax for the examples:

1. Filter out any failures on resources tagged with Name a.domain.com OR b.domain.com OR c.domain.com:

"some_prowler_check":
  Regions:
    - "*"
  Resources:
    - "*"
  Tags:
    - "Name=a.domain.com|Name=a.domain.com|Name=a.domain.com"

2. Mute for any host that has as Name any of the values as the example above, OR Type someType1 OR someType2:

"some_prowler_check":
  Regions:
    - "*"
  Resources:
    - "*"
  Tags:
    - "Name=a.domain.com|Name=a.domain.com|Name=a.domain.com|Type=someType1|Type=someType1"

Is that approach better for your use case?