Alternatives for setuptools.config.read_configuration (for both setup.cfg and pyproject.toml)

[flisboac]

I updated to Setuptools > 61.0.0, and now I see the following message when trying to use the aforementioned function:

setuptools\config\__init__.py:28: SetuptoolsDeprecationWarning: As setuptools moves its configuration towards `pyproject.toml`,
`setuptools.config.parse_configuration` became deprecated.

For the time being, you can use the `setuptools.config.setupcfg` module
to access a backward compatible API, but this module is provisional
and might be removed in the future.

  warnings.warn(dedent(msg), SetuptoolsDeprecationWarning)

from setuptools.config.setupcfg import read_configuration works, but cannot be found by linters when statically analyzing the file (at least pylance couldn't), so the import is "squiggled" and read_configuration is effectively Any.

I don't understand the rationale behind this decision. Why remove setup.cfg functionality while pyproject.toml-equivalent ones are still experimental (as per the documentation)?

Also, even when changing to setuptools.config.setupcfg.read_configuration, the warning message says the module is provisional and should not be depended upon, but no other option is given to users that depend on this feature. What sould I do instead? Or what can I use in its place?

In any case, I'm not sure if there's a read_configuration that's able to read pyproject.toml files with setuptools semantics (ie. loading a Distribution object, and all of that). I tried to read a pyproject.toml (converted from my setup.cfg via ini2toml) with read_configuration, and it failed, complaining that the file format is invalid:

Traceback (most recent call last):
  File "setup.py", line 9, in <module>
    _project_info = ProjectInfo(_project_root)
  File "(...)\project_info.py", line 29, in __init__
    self.config: Final = ProjectInfoConfig(project_root)
  File "(...)\project_info.py", line 310, in __init__
    self.config_source: Final[Mapping[str, Any]] = read_configuration(
  File "(...)\Python38\site-packages\setuptools\config\setupcfg.py", line 57, in read_configuration
    handlers = _apply(dist, filepath, filenames, ignore_option_errors)
  File "(...)\Python38\site-packages\setuptools\config\setupcfg.py", line 88, in _apply
    _Distribution.parse_config_files(dist, filenames=filenames)
  File "(...)\Python38\site-packages\setuptools\_distutils\dist.py", line 407, in parse_config_files
    parser.read(filename)
  File "(...)\Python38\Lib\configparser.py", line 697, in read
    self._read(fp, filename)
  File "(...)\Python38\Lib\configparser.py", line 1113, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: '(...)\\pyproject.toml'
        [line 15]: ']\n'
        [line 21]: ']\n'
        [line 43]: ']\n'

(project_info.py is a helper module I'm using as an external library for other scripts; it uses read_configuration to get the distribution's information)

What is the equivalent of read_configuration for pyproject.toml files?

[abravalheri]

Hi Flávio thank you very much for reaching out.

I am curious to understand what is your use case. Is it possible to elaborate a little about that?

The main rationale behind the deprecation is summarised in #3214. Since the long term plan is to remove setup.cfg all together and rely on "transpiling" it to TOML, it makes sense to let people currently relying on setuptools internal functions to parse setup.cfg that it will eventually go away.

Both setup.cfg and pyproject.toml are declarative files and can be parsed directly (e.g. via ConfigParser or tomli). My personal opinion is that the functions that setuptools uses internally to parse these files should remain private and not part of the exposed API.

[flisboac]

Hello, @abravalheri ! Thank you for the response!

I'm writing libraries and external scripts to automate some build/development time tasks, like provisioning virtual envs, initializing development environments, generating constraints files, and other assorted things. Currently, all of my stack is based on Pip and Setuptools. That project_info.py is responsible for providing project metadata, which as of now is read through read_configuration. From the project_info's perspective, I'd like to know the project's name, description, version, dependencies and anything else that may be described/normalized by the build system.

For now, I depend exclusively on pip/setuptools, but in the future, I'd like to evolve the solution to support other build systems, and enable its use in other projects. This requires reading pyproject.toml first to identify which build system is being used, so that I can select a proper or official metadata reader (ie. one that's provided by the build system itself) and/or determine how to interact with the project (e.g. which executable to use for some distribution operations). In setuptools' case, the executable is the project's setup.py, which may not exist, so there's no way to reliably use an external tool to extract project metadata. Therefore, I'm forced to use a more programmatic approach to reading project metadata (ie. use some library, read a configuration file, etc).

Also, for some projects, I need to dynamically construct extras_require, and that's only possible when using setup.py, so executing setup.py to obtain project metadata is out of the question. For those cases, I'm counting on setup.cfg being always present, so that I can read anything that's not requirements from it. The requires themselves will be on some other files of the project (ex. requirements/*.txt, etc), and my utility will help when constructing those extra requires' sets, among other things.

Ideally, I'd like to use setuptools whenever possible when a project depends on it; but at the same time, I would like to not rely too much on setuptools to build my utilities, because I cannot enforce its use on every project.

Migrating from setup.cfg right now is not possible (because I don't have enough time to do so, for all of my projects), but it's something I'll do in the future. I'm also not confident enough on editable installs with pyproject.toml to work everywhere (Windows, Linux, etc) like it does with setup.cfg (even with setup.py present; what I gather from some issues is that it's not truly stable, and there'll be some time until the rest of the community catches up with the recent changes in setuptools; therefore, setup.cfg still has some life in it, and it should be premature to deprecate it this early -- including the functions that load/read it, for external users).

---

AFAICT, pyproject.toml is standard TOML, so I may be able to read it with some third-party library, but setup.cfg has some syntax peculiarities, and I doubt I'd be able to read it with something external to setuptools. In this case, should I use ConfigParser instead? (Btw, where is it located? setuptools.config?)

I don't understand why read_configuration not part of the exposed API, when things like distutils extensions depend on Distribution (from Command.distribution), which is what read_configuration ultimately produces. Perhaps there should be another entrypoint for this use case? My line of thinking is that it makes sense to provide such a function, so that common attributes could be standardized as a proper class (ie. Distribution), regardless of file format, or future changes in the supported pyproject.toml properties, etc. I think it's unfair to lead users into guesswork when changes on pyproject properties (tool.setuptools.*, etc) are introduced.

[abravalheri]

Hi Flávio, thank you very much for the description of the use case.

Based on it, may I suggest an alternative approach that is also setuptools-independent?

The most reliable, non-backend-specific way of obtaining the information you are after seems to be using the core-metadata produced by the project.

Currently the pep517 project implements this via the pep517.meta.load function that could be used. However it is not clear from their project website if this function is part of the public API, so it is worthy to contact the project maintainers and ask.

Otherwise you could still do it yourself with importlib_metadata.Distribution.at + <build-backend>.prepare_metadata_for_build_wheel¹ but that is more complicated...

---

should I use ConfigParser instead? (Btw, where is it located? setuptools.config?)

ConfigParser is part of the standard library from configparser import ConfigParser.
According to the use case description I can see that the peculiarities would be:

• install_requires, extra_requires - with ConfigParser they would be given as a string and you would have to split them by line/;
• version, description, etc - with ConfigParser you would have to check if the value start with file: or attr: and process the value accordingly.

My line of thinking is that it makes sense to provide such a function, so that common attributes could be standardized as a proper class (ie. Distribution), regardless of file format, or future changes in the supported pyproject.toml properties, etc. I think it's unfair to lead users into guesswork when changes on pyproject properties (tool.setuptools.*, etc) are introduced.

I agree with you here that it makes sense, but I don't think setuptools is where this should happen. Indeed the alternative that I shared with you above is a standardised way of obtaining this information that pretty much fulfils this description.

Footnotes

1. https://pep517.readthedocs.io/en/latest/callhooks.html#pep517.Pep517HookCaller.prepare_metadata_for_build_wheel ↩