Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update of DPC dependencies for CI and conda-recipe #2160

Merged
merged 18 commits into from
Nov 14, 2024
Merged

Update of DPC dependencies for CI and conda-recipe #2160

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
3204792
Update of conda-recipe building dependencies
Alexsandruss Nov 11, 2024
f68ea7a
Update DPC deps in CI
Alexsandruss Nov 11, 2024
49b9e82
Update DPC compiler version to 2025
Alexsandruss Nov 11, 2024
5474ded
Change compiler installation to conda-forge
Alexsandruss Nov 11, 2024
6a7a8bd
Reorder dpep pkgs installation
Alexsandruss Nov 11, 2024
90ca9cb
Move dpcpp-cpp-rt from running deps to optional test deps
Alexsandruss Nov 11, 2024
f384680
Remove duplicated installation of numpy
Alexsandruss Nov 12, 2024
c885723
Change dpep installation to PyPI
Alexsandruss Nov 12, 2024
6ecd411
Change mpich to impi-devel in lnx CI
Alexsandruss Nov 12, 2024
248abbd
Revert dpc compiler installation
Alexsandruss Nov 13, 2024
4c112e6
Update conda recipe
Alexsandruss Nov 13, 2024
67e5430
Fix conda-recipe
Alexsandruss Nov 13, 2024
7bdafbc
Modify py versions for dpep installation
Alexsandruss Nov 13, 2024
6615964
Use dpcpp-rt from apt installation
Alexsandruss Nov 13, 2024
a30692a
Revert "Use dpcpp-rt from apt installation"
Alexsandruss Nov 13, 2024
6df1e68
Remove py312 for dpep installation
Alexsandruss Nov 13, 2024
e97c607
Revert mpi to impi
Alexsandruss Nov 14, 2024
664f3ed
Add comment about dpep installation
Alexsandruss Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 7 additions & 10 deletions .ci/pipeline/build-and-test-lnx.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,27 +16,24 @@
steps:
- script: sudo apt-get update && sudo apt-get install -y clang-format
displayName: "apt-get"
- script: |
bash .ci/scripts/install_dpcpp.sh
displayName: "dpcpp installation"
- script: |
source /opt/intel/oneapi/compiler/latest/env/vars.sh
bash .ci/scripts/describe_system.sh
displayName: "System info"
- script: |
conda config --add channels conda-forge
conda config --set channel_priority strict
conda update -y -q conda
conda create -q -y -n CB -c conda-forge python=$(PYTHON_VERSION) dal-devel mpich pyyaml "dpcpp-cpp-rt=2024.2.0"
conda create -q -y -n CB -c conda-forge python=$(PYTHON_VERSION) dal-devel=2025 mpich pyyaml dpcpp_linux-64=2025.0 dpcpp-cpp-rt=2025.0
displayName: "Conda create"
- script: |
. /usr/share/miniconda/etc/profile.d/conda.sh
conda activate CB
bash .ci/scripts/describe_system.sh
displayName: "System info"
- script: |
. /usr/share/miniconda/etc/profile.d/conda.sh
conda activate CB
pip install -r dependencies-dev
pip list
displayName: "Install develop requirements"
- script: |
export DPCPPROOT=/opt/intel/oneapi/compiler/latest
. /usr/share/miniconda/etc/profile.d/conda.sh
conda activate CB
export DALROOT=$CONDA_PREFIX
Expand All @@ -48,7 +45,7 @@ steps:
bash .ci/scripts/setup_sklearn.sh $(SKLEARN_VERSION)
pip install --upgrade -r requirements-test.txt
pip install $(python .ci/scripts/get_compatible_scipy_version.py)
if [ $(echo $(PYTHON_VERSION) | grep '3.9\|3.11') ] && [ $(SKLEARN_VERSION) != "1.0" ]; then conda install -q -y -c https://software.repos.intel.com/python/conda/ dpctl=0.17.0 dpnp=0.15.0; fi
if [ $(SKLEARN_VERSION) != "1.0" ]; then pip install dpctl==0.18.* dpnp==0.16.*; fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we installing dpctl/dpnp for all py versions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it has same supported python versions as sklearnex.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that does, however, cause certain functionality within sklearnex to become untested (see DummySyclQueue)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is DummySyclQueue needed if dpctl supports all sklearnex versions?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DummySyclQueue is needed for device offloading, if dpctl is not available in env.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there circumstances at this point where dpctl would not be available?

Copy link
Contributor

@samir-nasibli samir-nasibli Nov 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can give at least two examples:

  1. Installation Issues: problems on the user’s side during the installation of dpctl.
  2. Environment Compatibility: for example, dpctl does not currently support Python 3.13. While scikit-learn already supports Python 3.13 and we may soon add it to our matrices, dpctl will not be available for this Python version at this time.

pip list
displayName: "Install testing requirements"
- script: |
Expand Down
2 changes: 1 addition & 1 deletion .ci/scripts/install_dpcpp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,5 @@ rm GPG-PUB-KEY-INTEL-SW-PRODUCTS-2023.PUB
echo "deb https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list
sudo add-apt-repository -y "deb https://apt.repos.intel.com/oneapi all main"
sudo apt-get update
sudo apt-get install -y intel-dpcpp-cpp-compiler-2024.2
sudo apt-get install -y intel-dpcpp-cpp-compiler-2025.0
sudo bash -c 'echo libintelocl.so > /etc/OpenCL/vendors/intel-cpu.icd'
4 changes: 3 additions & 1 deletion conda-recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ requirements:
build:
- make # [linux]
- dpcpp_linux-64 # [linux64]
- kernel-headers_linux-64 ==4.18.0 # [linux64]
# - dpcpp_win-64 # [win]
- {{ compiler('cxx') }} # [linux64 or win]
# conda-forge feedstock specific
Expand All @@ -66,7 +67,6 @@ requirements:
run:
- python
- {{ pin_compatible('numpy') }}
- dpcpp-cpp-rt # [linux64]
# dal pinning depends on the recipe location (repo or feedstock)
- dal
# - dal =={{ version }}
Expand All @@ -75,6 +75,8 @@ test:
requires:
- pyyaml
- impi_rt # [not win]
# DPC part of sklearnex is optional
- dpcpp-cpp-rt # [linux64]
# TODO: enable data parallel frameworks when they are available on conda-forge
# - dpctl
# - dpnp
Expand Down
Loading