Running basedmypy with multiple configurations

[Zeckie]

Is there currently any way of setting up multiple configurations for a project (other than having 2 configuration files and using --config-file to select which to use)?

Some situations where I think this may be useful:

• When getting started (or moving between mypy and basedmypy), to help evaluate where the project is currently up to: Have a few builtin configurations (eg. different strictness ranging from optional typing up to strictest), run with each configuration, and give a summary of how many errors were reported at each strictness.
• When moving between configurations: Run with 2 configurations - a base (that the project already meets) and a target. Fail if any errors at the base configuration. For the target, use the baseline file as it currently is, to ignore existing errors.
• Running in different environments (operating system, python version, dependencies)

Some things to consider:

• How should those be stored in config file, so as not to interfere with mypy, if that is also used (eg. use [tool.basedmypy] instead of [tool.mypy] for any basedmypy specific configuration)?
• Could / should each configuration have its own cache, baseline file etc?
• Would command line switches affect all profiles?
• If run with command line switches that affect the ability to reuse cache, should it not use the cache?

[KotlinIsland]

A strategy that works quite well when dealing with migrating config options is to set your config file with the current strictness you have:

[tool.mypy]
disallow_any_generics = true

Then invoke mypy with --legacy, this will set the default options to un-strict but still use the options specified in the config file:

> mypy --legacy --baseline-file= src
...
> mypy src
...

Running in different environments (operating system, python version, dependencies)

That's more related to python/issues/12286 I think.

I think this strategy covers the first two options fairly well, but would be based if this multi run happened by default, and maybe a config to disable it.

Is there any use-case for different configuration profiles other than these three?

How should those be stored in config file

Mypy ignores unrecognized configuration options with a warning #158

[Zeckie]

But doesn't that have issues like:

• it using the same cache, so will clear that cache when run with the different configurations
• deleting the baseline file due to there being no errors when run with --legacy (auto_baseline should do better #261 (comment))?
• doesn't legacy use the same config file (so if you add something to the config, it applies whether or not --legacy is used)?

It also requires running basedmypy multiple times, when it would be useful to run it once and have it check both configs.

[Zeckie]

But yes, I guess I could solve those issues by writing my own script, to run basedmypy twice, use separate caches, use a different configuration file, and use baseline for the target config only.

[KotlinIsland]

deleting the baseline file due to there being no errors when run with --legacy

Specify no baseline file with --baseline-file=

doesn't legacy use the same config file (so if you add something to the config, it applies whether or not --legacy is used)?

For this strategy/scenario where you only have two configurations, a pre-baseline, and a stricter + baseline, using legacy handles it correctly.

As I asked before, do you have any use-case outside of this scenario?

would be based if this multi run happened by default

I think there would be good value adding this functionality, I'm just wondering what scope it should have.