Superset config_path and plugin definition schema change process

[pnadolny13]

I was attempting to use the config_path setting to pass my custom Superset config and I got tripped up and was about to file a bug when I realized the setting is at the plugins top level with the pip_url and name, not under config like I would have expected. To be fair the hub docs do say it clearly on a second look but I missed it the first time. I also get schema validation errors in VS code because its not listed in https://github.com/meltano/meltano/blob/e02c48b0ca676745a5a4db934702242c1e501a8a/schema/meltano.schema.json#L109. Is this intended to be generically useful, to me initially it seems like a Superset specific setting that shouldnt get mixed in with the common plugin attributes.

I'm not sure if theres precedence for adding new keys at the plugin top level, but was there a larger conversation about adding it there? Could it be moved to a setting? Whats our process for adding new keys to plugin definitions?

@DouweM @aaronsteers What do you think?

[DouweM]

@pnadolny13 config_path is a plugin extra, like select and catalog on Singer taps. It's meant to be used (as written in that doc) for options that are handled by Meltano instead of the plugin itself. Here my reasoning was that everything under config is passed verbatim to Superset, and a property we handle should therefore not go into config. Having config_path at the same level as config also made aesthetic sense to me.

This config_path is somewhat similar to dbt's project_dir, which is a regular setting rather than an extra, but that's because it's still directly passed to dbt on the command line or through an environment variable rather than being pre-processed by Meltano itself.

It is currently Superset specific, but we can add it to Singer and Airlfow as well: meltano/meltano#6045. Basically to allow a pre-built config file to be provided to any plugin, to circumvent or extend the Meltano-managed config.

I'm not sure we can teach the JSONSchema validation to only allow this property when name == "superset".