[CI][Python] Investigate trusted publishing for uploading wheels to PyPI

[pitrou]

Describe the enhancement requested

"Trusted publishing" may help improve the security of wheel uploads, and also provide automated digital attestations.
See https://discuss.python.org/t/pypi-now-supports-digital-attestations/71158 and https://trailofbits.github.io/are-we-pep740-yet/ .

It might make it either easier or harder to publish releases, however :)

Component(s)

Continuous Integration, Packaging, Python

[pitrou]

cc @raulcd @assignUser

[assignUser]

We talked about trusted publishing and related GitHub features just yesterday at the infra roundtable!

[pitrou]

Ah, it's nice that you attend those. Thanks a lot :)

[raulcd]

I am curious on how the ASF release process wants to handle those. Currently if we follow the current release process we would be generating the binaries, uploading them to dist.apache.org, verifying them and once the vote passes, downloading them locally from dist and uploading them to PyPI.
If we still want to generate binaries and "vote" them, we still would have to upload them to dist.apache.org and after the vote passes create a GH action that downloads the "voted" binaries and uploads those to PyPI, while generating the short lived tokens for "Trusted publishing" this would remove the possibility of an ill intentioned PMC or committer with rights to upload wheels from uploading wrong wheels but there is still the case of someone uploading wrong wheels to dist.apache.org.

A different solution could be to just generate the final wheels once the source has been voted and released and upload those newly generated wheels on the same job that generates them without possibility of tampering (bear in mind source code is signed and could be validated that no modification has happened).
I am curious on what other projects do, are other projects not voting on wheels and just publishing afterwards or are they downloading the generated wheels and uploading them as we do?

[raulcd]

cc @kou for feedback too

[pitrou]

I've often said that I don't find our voting process useful. If the binaries are thoroughly verified on CI jobs, then it does not really make sense to have them verified manually, either (especially as the manual verification just consists in executing a script).