Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use meson build in place of numpy distutils #12

Merged
merged 54 commits into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
a70e5b3
first attempt at meson building
jcapriot Nov 20, 2023
c0eee6c
include mkl-devel in build requirements
jcapriot Nov 21, 2023
a12b70c
change version
jcapriot Nov 21, 2023
fd02e86
relax version a bit
jcapriot Nov 21, 2023
c10396c
use examples from numpy
jcapriot Nov 22, 2023
63b8c86
update github workflow, reenable thread locks
jcapriot Jan 8, 2024
684f90e
Merge branch 'meson-build' of https://github.com/jcapriot/pydiso into…
jcapriot Jan 8, 2024
2d2de6c
update meson build file for mkl linkage parameters
jcapriot Jan 8, 2024
d558c56
remove print outs
jcapriot Jan 8, 2024
c430f1f
remove setup.py file
jcapriot Jan 8, 2024
f462fde
install setuptools
jcapriot Jan 8, 2024
368508a
add option to force sdl usage
jcapriot Jan 8, 2024
d635c8a
do not fail fast on tests
jcapriot Feb 28, 2024
d45857b
move thread lock allocation
jcapriot Feb 28, 2024
4c9a9d7
update some meta information
jcapriot Feb 29, 2024
6d11442
ignore pycharm files
jcapriot Feb 29, 2024
4ed000d
be a bit more accurate about the argument types
jcapriot Feb 29, 2024
d762962
Be a little safer about free the lock
jcapriot Feb 29, 2024
5fa9ade
split __dealloc__ into a dealloc and a __del__ method for safety
jcapriot Feb 29, 2024
bdd2da7
determine initialization status dynamically
jcapriot Feb 29, 2024
e52dc86
make _initialized a function
jcapriot Feb 29, 2024
8b967a6
just use if self.handle[i]
jcapriot Feb 29, 2024
bbcdbd5
use mamba
jcapriot Feb 29, 2024
a567a73
cache the environment
jcapriot Feb 29, 2024
b233eed
fix indices and indptr
jcapriot Mar 1, 2024
d018d0b
stop overwriting dtype
jcapriot Mar 1, 2024
88f7ea8
undo some types
jcapriot Mar 1, 2024
5deb393
more type resetting
jcapriot Mar 1, 2024
9c1ac26
unsetting some more
jcapriot Mar 1, 2024
0303889
make call type a little more accurate
jcapriot Mar 1, 2024
41f85a4
switch back to conda (with defaults)
jcapriot Mar 1, 2024
a41b8d6
reset some build scripts
jcapriot Mar 1, 2024
ab72d25
switch back to mamba, but don't use mkl 2024
jcapriot Mar 1, 2024
8765c59
ensure cleared cache
jcapriot Mar 1, 2024
9e72e6e
retry this
jcapriot Mar 1, 2024
6906999
dynamic initialized (again)
jcapriot Mar 1, 2024
6f03c16
be explicit about initializing handle and iparm
jcapriot Mar 1, 2024
25ac2f5
don't cache
jcapriot Mar 2, 2024
207dc8e
generate c sources in tree for cython coverage
jcapriot Mar 11, 2024
00435b1
enable coverage build option
jcapriot Mar 11, 2024
29bd20a
have meson conditionally generate c file in source directory when doi…
jcapriot Mar 11, 2024
24498d6
add coverage configuration to pyproject.toml
jcapriot Mar 11, 2024
a8e5418
remove all pytest-cov related options to pytest
jcapriot Mar 11, 2024
d1a0be5
fix coverage determination, split long lines
jcapriot Mar 11, 2024
596ed7a
generate coverage.xml
jcapriot Mar 11, 2024
2aeceaf
fix command
jcapriot Mar 12, 2024
cab5d60
add comments to meson build file
jcapriot Mar 13, 2024
f492847
update distribution script
jcapriot Mar 13, 2024
ed93d8e
update gitignore
jcapriot Mar 13, 2024
ef07d20
add a test for mkl=2024
jcapriot Mar 13, 2024
0fbd9b4
update test name
jcapriot Mar 13, 2024
3ba4c88
fix matrix variable name
jcapriot Mar 13, 2024
de6d10b
add test for windows and mac on mkl 2024
jcapriot Mar 13, 2024
953be5e
mkl-2024 is not on macos
jcapriot Mar 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .bumpversion.cfg

This file was deleted.

5 changes: 0 additions & 5 deletions .coveragerc

This file was deleted.

4 changes: 4 additions & 0 deletions .git_archival.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
node: $Format:%H$
node-date: $Format:%cI$
describe-name: $Format:%(describe:tags=true,match=*[0-9]*)$
ref-names: $Format:%D$
11 changes: 11 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
.git_archival.txt export-subst
# Excluding files from an sdist generated by meson-python

.azure-pipelines/* export-ignore
.ci/* export-ignore
tests/* export-ignore

.flake8 export-ignore
.git* export-ignore
*.yml export-ignore
*.yaml export-ignore
82 changes: 47 additions & 35 deletions .github/workflows/python-package-conda.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,61 +14,71 @@ on:

jobs:
build-and-test:
name: Testing (${{ matrix.python-version }}, ${{ matrix.os }})
name: Testing (Python ${{ matrix.python-version }}, on ${{ matrix.os }}, with MKL ${{ matrix.mkl-version }})
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash -l {0}
strategy:
fail-fast: True
fail-fast: False
matrix:
os: [ubuntu-latest, macOS-latest, windows-latest]
python-version: ["3.8", "3.9", "3.10", "3.11"]
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
mkl-version: ['2023'] # currently 2024 fails building for some reason...
include:
- os: ubuntu-latest
python-version: "3.12"
coverage: ${{ true }}
- os: ubuntu-latest
python-version: "3.12"
mkl-version: '2024'
- os: windows-latest
python-version: "3.12"
mkl-version: '2024'

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Setup Conda
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
auto-update-conda: true
activate-environment: dev
python-version: ${{ matrix.python-version }}
- name: Install Env
mamba-version: '*'
channels: conda-forge, defaults
channel-priority: true
activate-environment: dev

- name: Conda information
run: |
conda info
conda list
conda config --show
conda install --quiet --yes pip numpy scipy cython mkl pytest pytest-cov mkl-devel

- name: Install Our Package
if: ${{ matrix.os != 'windows-latest' }}
- name: Create environment
run: |
echo $CONDA_PREFIX
export TEST_COV=1
pip install -v -e .
conda list
mamba install --quiet --yes pip numpy scipy cython mkl=${{ matrix.mkl-version }} pytest \
mkl-devel pkg-config meson-python meson ninja setuptools_scm \
${{ matrix.coverage && 'coverage' || ''}}

- name: Install Our Package on Windows
if: ${{ matrix.os == 'windows-latest' }}
- name: Install Our Package
run: |
echo $CONDA_PREFIX
export MKLROOT="$CONDA_PREFIX\Library"
echo $MKLROOT
pip install -v -e .
python -m pip install --no-build-isolation --verbose --editable . \
--config-setting=compile-args=-v \
${{ matrix.coverage && '--config-settings=setup-args="-Dcy_coverage=true"' || ''}}
conda list

- name: Run Tests
run: |
pytest --cov-config=.coveragerc --cov-report=xml --cov=pydiso -s -v
${{ matrix.coverage && 'coverage run -m' || '' }} pytest -s -v
${{ matrix.coverage && 'coverage xml' || '' }}

- name: Upload coverage
if: ${{ matrix.os == 'ubuntu-latest' }} and {{ matrix.python-version == '3.8' }}
uses: codecov/codecov-action@v2
if: ${{ matrix.coverage }}
uses: codecov/codecov-action@v4
with:
verbose: true # optional (default = false)

distribute:
name: Distributing from 3.8
name: Distributing from 3.11
needs: build-and-test
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
runs-on: ubuntu-latest
Expand All @@ -77,29 +87,31 @@ jobs:
shell: bash -l {0}

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Setup Conda
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
auto-update-conda: true
python-version: 3.11
mamba-version: '*'
channels: conda-forge, defaults
channel-priority: true
activate-environment: dev
python-version: 3.8

- name: Install Env
run: |
conda info
conda list
conda config --show
conda install --quiet --yes pip numpy scipy cython mkl mkl-devel

- name: Install Our Package
- name: Create environment
run: |
echo $CONDA_PREFIX
pip install -v -e .
mamba install --quiet --yes pip numpy scipy cython mkl=2023 \
mkl-devel pkg-config meson-python meson ninja setuptools_scm \
python-build

- name: Generate Source Distribution
run: |
python setup.py sdist
python -m build --no-isolation --skip-dependency-check --sdist .

- name: pypi-publish
uses: pypa/gh-action-pypi-publish@v1.4.2
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
build
dist
coverage_html_report
*.so
pydiso.egg-info
__pycache__
mkl_solver.c
mkl_solver.c.dep
.coverage
coverage.xml

.idea/
3 changes: 0 additions & 3 deletions MANIFEST.in

This file was deleted.

27 changes: 10 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,29 @@
# pydiso

Pydiso is a python wrapper for the pardiso solver. It is currently implemented for the
Intel MKL's version of pardiso. It's goal is to expose the full functionality of pardiso
Intel MKL's version of pardiso. Its goal is to expose the full functionality of pardiso
to python, allowing the user to customize it as necessary for their use cases.

# Installation


### Installing from source
## Installing from source

The wrapper is written in cython and links to the mkl libraries dynamically. Therefore,
it needs to find the necessary header files associated with the MKL installation to compile.
For conda users, these headers can be installed with `mkl-devel` package that is available
on the default channel, conda-forge channel, the intel channel, or others, e.g.
The meson build backend uses pkg-config to identify the locations of the mkl header files
and library dynamic libraries. Most development installations of MKL should provide the
necessary pkg-config files for this. For example, conda users can be install the necessary
configuration information with `mkl-devel` package that is available on the default channel,
conda-forge channel, the intel channel, or others, e.g.

`conda install mkl-devel`

Most of the time, your respective numpy installations will then be intelligent enough to
identify the location of the installed MKL. However if you run into issues automatically
finding the library headers, you will need to set the `MKLROOT` environment variable to
point to the correct location. On Linux and MacOS the `mkl-rt` library and `mkl.h` are
usually in the same folder, and thus the root should point to that directory. On Windows
the `mkl-rt.h` and `mkl-rt.lib` are not in the same folder but seperated a level into
`Library` and `Include` directories, and in this case `MKLROOT` would point to the folder
containing them.
If you have installed the configuration files to a non-standard location, you will need to set
`PKG_CONFIG_PATH` to point to that location.

After the necessary MKL files are accessible, you should be able to do the standard install
script common to python packages by running either

`python setup.py install`

or, equivalently
After the necessary MKL files are accessible, you should be able to install by running

`pip install .`

Expand Down
50 changes: 50 additions & 0 deletions meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
project(
'pydiso',
'c', 'cython',
# Note that the git commit hash cannot be added dynamically here
# (it is dynamically generated though setuptools_scm)
version: run_command('python',
[
'-c',
'''
from setuptools_scm import get_version
print(get_version())'''
],
check: true
).stdout().strip(),

license: 'MIT',
meson_version: '>= 1.1.0',
default_options: [
'buildtype=debugoptimized',
'b_ndebug=if-release',
],
)

# https://mesonbuild.com/Python-module.html
py_mod = import('python')
py = py_mod.find_installation(pure: false)
py_dep = py.dependency()

cc = meson.get_compiler('c')
cy = meson.get_compiler('cython')
# generator() doesn't accept compilers, only found programs - cast it.
cython = find_program(cy.cmd_array()[0])

_global_c_args = cc.get_supported_arguments(
'-Wno-unused-but-set-variable',
'-Wno-unused-function',
'-Wno-conversion',
'-Wno-misleading-indentation',
)
add_project_arguments(_global_c_args, language : 'c')

# We need -lm for all C code (assuming it uses math functions, which is safe to
# assume for SciPy). For C++ it isn't needed, because libstdc++/libc++ is
# guaranteed to depend on it.
m_dep = cc.find_library('m', required : false)
if m_dep.found()
add_project_link_arguments('-lm', language : 'c')
endif

subdir('pydiso')
10 changes: 10 additions & 0 deletions meson.options
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
option('cy_coverage', type : 'boolean', value : false)

option('use-sdl', type: 'boolean', value: true,
description: 'Use the single dynamic library.')

option('use-ilp64', type: 'boolean', value: false,
description: 'Use ILP64 (64-bit integer) BLAS and LAPACK interfaces')

option('mkl-threading', type: 'string', value: 'auto',
description: 'MKL threading method, one of: `seq`, `iomp`, `gomp`, `tbb`')
17 changes: 16 additions & 1 deletion pydiso/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,19 @@
__version__ = "0.0.3"
__author__ = "SimPEG Team"
__license__ = "MIT"
__copyright__ = "2021, SimPEG Developers, http://simpeg.xyz"

from importlib.metadata import version, PackageNotFoundError

# Version
try:
# - Released versions just tags: 0.8.0
# - GitHub commits add .dev#+hash: 0.8.1.dev4+g2785721
# - Uncommitted changes add timestamp: 0.8.1.dev4+g2785721.d20191022
__version__ = version("pydiso")
except PackageNotFoundError:

Check warning on line 13 in pydiso/__init__.py

View check run for this annotation

Codecov / codecov/patch

pydiso/__init__.py#L13

Added line #L13 was not covered by tests
# If it was not installed, then we don't know the version. We could throw a
# warning here, but this case *should* be rare. discretize should be
# installed properly!
from datetime import datetime

Check warning on line 17 in pydiso/__init__.py

View check run for this annotation

Codecov / codecov/patch

pydiso/__init__.py#L17

Added line #L17 was not covered by tests

__version__ = "unknown-" + datetime.today().strftime("%Y%m%d")

Check warning on line 19 in pydiso/__init__.py

View check run for this annotation

Codecov / codecov/patch

pydiso/__init__.py#L19

Added line #L19 was not covered by tests
Loading
Loading