chore: use package_distributions to get top levels

[mkniewallner]

PR Checklist

• A description of the changes is added to the description of this PR.
• If there is a related issue, make sure it is linked to this PR.
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added or modified a feature, documentation in docs is updated

Description of changes

Those changes are extracted from #854, as a first step towards using importlib.metadata/importlib_metadata to remove some custom code we have, and make "package" <-> "distribution" discovery more precise.

Python 3.10 introduced packages_distributions that returns a mapping that maps Python module names to the packages that expose them, by reading from top_level.txt like we do. In Python 3.11, the method became even smarter by falling back to reading from RECORD when no top_level.txt is present, also exactly like we do.

Updates in importlib.metadata are upstreamed from importlib_metadata, so by requiring >=4.13 on Python < 3.11, we could use packages_distributions in all Python versions we support, and also benefit from the improvement made on Python 3.11.

There is one functional change in behaviour with the new implementation, which IMO makes sense to have. Currently, if a package has a RECORD file, but we don't match any Python module name in the file, we don't use the fallback that assumes the module name is the name with _ instead of -. With the changes, we will now use the fall back if there was no match in RECORD.

Test plan

• Existing functional tests are passing
• Functional test added to ensure that package normalisation is applied
• Added unit tests for the newly added functions

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.7%. Comparing base (6d9f398) to head (0640431).

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #861     +/-   ##
=======================================
+ Coverage   92.1%   92.7%   +0.6%     
=======================================
  Files         37      39      +2     
  Lines        992     993      +1     
  Branches     175     173      -2     
=======================================
+ Hits         914     921      +7     
+ Misses        62      57      -5     
+ Partials      16      15      -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fpgmaas]

At first glance it is a bit confusing to understand the difference between this function and get_normalized_distributions_packages. Maybe we can make their names slightly more verbose? Or add a small example output in their docstring?

Suggested change

	def get_packages_normalized_distributions() -> dict[str, set[str]]:
	def get_packages_to_normalized_distributions_mapping() -> dict[str, set[str]]:

[fpgmaas]

Suggested change

	def get_normalized_distributions_packages() -> dict[str, set[str]]:
	def get_normalized_distributions_to_packages_mapping() -> dict[str, set[str]]: