Migrate away from the deprecated pkgutil.find_loader()

[edmorley]

In the newly released Python 3.12, the pkgutil.find_loader() API was deprecated:
https://docs.python.org/3.12/library/pkgutil.html#pkgutil.find_loader

This API was used by the buildpack to determine whether certain packages (like django or nltk) are installed.

It's been replaced by importlib.util.find_spec which has been supported since Python 3.4 (which is fine for us to use now, but wasn't an option back when is_module_available() was written, since at that time the buildpack still had to support Python 2.7):
https://docs.python.org/3.12/library/importlib.html#importlib.util.find_spec

Using the old API results in deprecation warnings being output in the build log:

<string>:1: DeprecationWarning: 'pkgutil.find_loader' is deprecated and slated for removal in Python 3.14; use importlib.util.find_spec() instead

These deprecation warnings also prevent the output from bin/release being parsed correctly for Django apps, resulting in:

failed to read buildpack metadata: (<unknown>): mapping values are not allowed in this context at line 1 column 31

These can be seen in the Heroku CI tests and Review App build logs of this PR which upgrades the Python getting started guide app to Python 3.12:
heroku/python-getting-started#209
https://dashboard.heroku.com/pipelines/4826c212-2bb2-4882-a147-4d0501eee7a4/tests/190
https://dashboard.heroku.com/apps/getting-star-edmorley-p-biivve/activity/builds/a6feca96-d451-4f01-b66d-c5d2185d5107

GUS-W-14230170.

[schneems]

Is the output of DeprecationWarning consistent in Python? We could add a refutation to integration tests to catch them earlier.

[edmorley]

Published:

$ make publish
Deploy as version: v237 [y/n]? y

Tagging commit 885ec9312b500ed84bbb6126a22b8171d61900fa with v237... 
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/heroku/heroku-buildpack-python
 * [new tag]         v237 -> v237

Publishing to the buildpack registry...
Two-factor code: ********************************************
Publishing buildpack heroku/python... done
Publishing buildpack was successful

 Version Released At              Status    
 ─────── ──────────────────────── ───────── 
 237     2023-10-03T20:43:07.366Z published

[edmorley]

Is the output of DeprecationWarning consistent in Python? We could add a refutation to integration tests to catch them earlier.

The tests in this repo use multi-line assertions that in many cases cover the majority of the build log output, so we'd catch spurious output in the build output in many cases (unless it appeared at the very end of the log etc).

The issue here is that whilst we test the buildpack against every single supported major Python version (and on every supported stack), we only do so for the basic tests, with the more niche buildpack features only being tested against the buildpack's default Python version. Running every test against all Python versions would increase CI time considerably, in return for catching a once in 4 year type deprecation warning.

For the CNB I've already stopped using Python APIs for this type of check to remove the chance of this scenario occurring. Perhaps it's one of the things I can backport (though needing to continue to support NLTK in the classic buildpack won't help, since the approach I used for Django detection can't be used for NLTK).