⭐ support querypacks + filter supported queries (#908)

* ⭐ support querypacks + filter supported queries

Requires mondoohq/cnquery#2389

1. Add support for querypacks
   - With cnspec building on top of cnquery, it was one of those
     features we have wanted to add for a while to make it more
     consistent
   - Querypacks will be executed and results printed to CLI. Note that
     we are in the process of improving the default output to be more
     compact, which will come in a follow-up soon.
2. Remove unsupported queries/checks
   - Users may not have all providers installed, which has caused
     wide-reaching policies (e.g. system-detection, or cloud+terraform
     hybrid policies) to throw errors
   - If a query or check cannot be compiled on local execution, it is
     now removed from the execution
   - Upstream policies continue to be handled in the usual way, since
     we only receive the subset that is applicable to a system
3. Add converter for querypacks to policies
   - This is more of an internal feature, but we wanted to make the code
     open to show how these two types are converted and then executed

This brings us in line with cnquery behavior of bundles.

Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

* 🟢 fix linter issues

Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

---------

Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

apps/cnspec/cmd/scan.go

@@ -238,7 +238,7 @@ func getCobraScanConfig(cmd *cobra.Command, runtime *providers.Runtime, cliRes *
  Features: opts.GetFeatures(),
  IsIncognito: viper.GetBool("incognito"),
  Inventory: inv,
- PolicyPaths: viper.GetStringSlice("policy-bundle"),
+ PolicyPaths: dedupe(viper.GetStringSlice("policy-bundle")),
  PolicyNames: viper.GetStringSlice("policies"),
  ScoreThreshold: viper.GetInt("score-threshold"),
  Props: props,
@@ -322,7 +322,14 @@ func (c *scanConfig) loadPolicies() error {
  return err
  }
 
- _, err = bundle.Compile(context.Background(), c.runtime.Schema(), nil)
+ _, err = bundle.CompileExt(context.Background(), policy.BundleCompileConf{
+ Schema: c.runtime.Schema(),
+ // We don't care about failing queries for local runs. We may only
+ // process a subset of all the queries in the bundle. When we receive
+ // things from the server, upstream can filter things for us. But running
+ // them locally requires us to do it in here.
+ RemoveFailing: true,
+ })
  if err != nil {
  return errors.Wrap(err, "failed to compile bundle")
  }
@@ -386,3 +393,15 @@ func printReports(report *policy.ReportCollection, conf *scanConfig, cmd *cobra.
  log.Fatal().Err(err).Msg("failed to print")
  }
 }
+
+func dedupe[T string | int](sliceList []T) []T {
+ allKeys := make(map[T]bool)
+ list := []T{}
+ for _, item := range sliceList {
+ if _, value := allKeys[item]; !value {
+ allKeys[item] = true
+ list = append(list, item)
+ }
+ }
+ return list
+}