Excluding Python source files from wheels generated by cibuildwheel

[cdhainaut]

Hi, I'm using GitHub Actions with cibuildwheel to build and publish my Python package, but I'd like to exclude the Python source code files (.py files) from the generated wheel files to protect the source code. Here’s my current workflow file:

name: Publish Package

on:
  push:
    tags:
      - "v*.*.*"
  workflow_dispatch:

jobs:
  build-and-publish:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.11"

      - name: Install build dependencies
        run: |
          python -m pip install --upgrade pip
          pip install cibuildwheel twine cython

      - name: Build wheels with cibuildwheel
        env:
          CIBW_BUILD: "cp311-*"
        run: |
          cibuildwheel --output-dir dist

      - name: Publish to PyPI
        if: github.ref_type == 'tag' || github.event_name == 'workflow_dispatch'
        env:
          TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
          TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
        run: |
          twine upload dist/*

The wheels build successfully and are published to PyPI, but they still contain the .py files. Could you suggest a way to configure cibuildwheel (or add additional steps) to exclude these .py files and only include the compiled .so (or .pyd on Windows) files?

Thank you !

[henryiii]

You are using very old versions of GitHub Actions. setup-python is on v5, for example. I'd also recommend the PyPA action for uploading instead of manually using twine (and using trusted publishers instead of tokens or especially username/password). Anyway, controlling what files go into your wheel is done by the backend. What backend are you using? Also, usually, you want both .py files and .so files unless they are tests or scripts or something like that.

See https://learn.scientific-python.org/development/guides/.

[cdhainaut]

Hi. Thanks for answering. I don't want my .py files... Thats the point. Only .so. I want to distribute a package without the source code. I wouldnt bother doing the compilation otherwise
Regarding the backend, I use setuptools.build_meta

Thank you

[henryiii]

How are you turning .py (python files) into .so (compiled C/C++/etc) code?

Or are you only trying to ship .pyc files? Those can be decompiled back to Python but provide a tiny bit of protection, at least for someone taking a quick glance.

If you do have .so’s from your .py’s, like mypyc, then you should be able to tell setuptools that you have no packages, turn off include-package-data, the manually list the .so file patters in package-data, I think. I’ve never tried it.

[cdhainaut]

Hi @henryiii

I use Cython in my setup.py to convert Python files into .so files with the following configuration:

ext_modules=cythonize(extensions, compiler_directives={"language_level": "3"}),
include_package_data=False,
zip_safe=False,

I’ve tried several approaches to exclude the Python files from the generated wheel, but none have worked. For example:

• Setting in pyproject.toml

[tool.setuptools]
package-dir = {"" = "."}
include-package-data = false

[tool.setuptools.packages.find]
where = ["."]
include = ["greety*"]

[tool.setuptools.exclude-package-data]
"*" = ["*.py"]

• Using MANIFEST.in

# Exclude all .py files from the final package
exclude *.py
recursive-exclude greety *.py

Despite all of this, I still see the Python files included in the final package.

Thanks for the support

[henryiii]

Oh, if it's Cython, can't you just name them .pyx?

You don't want to remove the .py files from the SDist (MANIFEST.in), just don't distribute an SDist.

You want to exclude the package, not find them ([tool.setuptools.packages.find]), and then add the .so/.pyd files via package-data, I believe.

[cdhainaut]

I'm not sure I understand. If I rename the .py files to .pyx, how will cibuildwheel compile them into .so files?

In my publish.yml:

      - name: Install build dependencies
        run: |
          python -m pip install --upgrade pip
          pip install cibuildwheel cython twine

      - name: Build wheels with cibuildwheel
        env:
          CIBW_BUILD: "cp311-*"
        run: |
          cibuildwheel --output-dir dist

I’m not executing any compilation commands before invoking cibuildwheel. It seems that cibuildwheel takes care of the compilation for each Linux platform and packages the result into a wheel. I'm unclear on how renaming the Python files to .pyx would allow this process to work
In the package (https://github.com/cdhainaut/GreeTy) you can see there are only python files at first. When published on PiPy I get a .so file inside each wheel but this is done through cibuilwheel...

EDIT: I see the py to pyx trick when I do manually the change of extension AND the renaming in the setup.py... But, my real package is fairly large, and, is used in hybrid contexts (protected and not). I would like to keep the setup.py unchanged and rather find a cibuildwheel option to discard these files, or a github action trick to rename them automatically (very bad option I think...)

Thank you

[cdhainaut]

Fixed setting packages parameter as empty list (packages=[]) in setup.py
Thanks @henryiii