Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TEST: added different dataframes testing in sklearnex own tests #1379

Merged
merged 16 commits into from
Aug 14, 2023

Conversation

samir-nasibli
Copy link
Contributor

@samir-nasibli samir-nasibli commented Jul 27, 2023

Description

Added different dataframes testing in sklearnex own tests for checking dpnp and dpctl interop in sklearnex.

  • _dataframes_support modul in onedal tests utils. Will reuse for onedal4py tests also.
  • Updated tests of modules:
    • sklearnex.neignbors
    • sklearnex.preview.decomposition
    • sklearnex.preview.ensemble
    • sklearnex.preview.linear_model
    • sklearnex.svm

Before:

neighbors/tests/test_neighbors.py::test_sklearnex_import_knn_classifier PASSED [  3%]

After:

neighbors/tests/test_neighbors.py::test_sklearnex_import_knn_classifier[numpy] PASSED [  3%]
neighbors/tests/test_neighbors.py::test_sklearnex_import_knn_classifier[dpctl-SyclQueue_CPU] PASSED [  4%]
neighbors/tests/test_neighbors.py::test_sklearnex_import_knn_classifier[dpctl-SyclQueue_GPU] PASSED [  4%]
neighbors/tests/test_neighbors.py::test_sklearnex_import_knn_classifier[dpnp-SyclQueue_CPU] PASSED [  4%]
neighbors/tests/test_neighbors.py::test_sklearnex_import_knn_classifier[dpnp-SyclQueue_GPU] PASSED [  4%]

TODO:

@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli
Copy link
Contributor Author

In this PR, the number of sklearnex own test cases has increased. Added dpnp and dpctl dataframes with cpu and gpu contexts to certain tests. So we have new 56 test cases, which checks dpnp/dpcttl interop.

Unfortunately, there are 18 new failures for different reasons, that should be addressed in separate PRs.

=========================== short test summary info ============================
FAILED neighbors/tests/test_neighbors.py::test_sklearnex_import_nn[dpctl-SyclQueue_CPU]
FAILED neighbors/tests/test_neighbors.py::test_sklearnex_import_nn[dpctl-SyclQueue_GPU]
FAILED neighbors/tests/test_neighbors.py::test_sklearnex_import_nn[dpnp-SyclQueue_CPU]
FAILED neighbors/tests/test_neighbors.py::test_sklearnex_import_nn[dpnp-SyclQueue_GPU]
FAILED preview/ensemble/tests/test_preview_ensemble.py::test_sklearnex_import_rf_regression[dpctl-SyclQueue_GPU]
FAILED preview/ensemble/tests/test_preview_ensemble.py::test_sklearnex_import_rf_regression[dpnp-SyclQueue_GPU]
FAILED preview/ensemble/tests/test_preview_ensemble.py::test_sklearnex_import_et_classifier[dpctl-SyclQueue_GPU]
FAILED preview/ensemble/tests/test_preview_ensemble.py::test_sklearnex_import_et_classifier[dpnp-SyclQueue_GPU]
FAILED preview/ensemble/tests/test_preview_ensemble.py::test_sklearnex_import_et_regression[dpctl-SyclQueue_GPU]
FAILED preview/ensemble/tests/test_preview_ensemble.py::test_sklearnex_import_et_regression[dpnp-SyclQueue_GPU]
FAILED svm/tests/test_svm.py::test_sklearnex_import_nusvc[dpctl-SyclQueue_GPU]
FAILED svm/tests/test_svm.py::test_sklearnex_import_nusvc[dpnp-SyclQueue_GPU]
FAILED svm/tests/test_svm.py::test_sklearnex_import_svc[dpctl-SyclQueue_GPU]
FAILED svm/tests/test_svm.py::test_sklearnex_import_svc[dpnp-SyclQueue_GPU]
FAILED svm/tests/test_svm.py::test_sklearnex_import_svr[dpctl-SyclQueue_GPU]
FAILED svm/tests/test_svm.py::test_sklearnex_import_svr[dpnp-SyclQueue_GPU]
FAILED svm/tests/test_svm.py::test_sklearnex_import_nusvr[dpctl-SyclQueue_GPU]
FAILED svm/tests/test_svm.py::test_sklearnex_import_nusvr[dpnp-SyclQueue_GPU]
===== 18 failed, 363 passed, 7 skipped, 429 warnings in 157.47s (0:02:37) ======

Currently, I have disabled this test cases via test filters + TODO comment, like we do for the tests for onedal module.
For example

# TODO:
# investigate failure for `dpnp.ndarrays` and `dpctl.tensors` on `GPU`
@pytest.mark.parametrize(
    "dataframe,queue", _get_dataframes_and_queues(device_filter_="cpu")
)
def test_sklearnex_import_et_classifier(dataframe, queue):

We don't have list of sklearnex or onedal deselected test like we have for sklearn tests, see deselected_tests.yaml.

I think we should extend sklearnex testing and add support of deselected tests for onedal4py and sklearnex modules instead of using filters. This is new feature, that should be implemented in separate branch. See TODO in [description] (#1379 (comment))

@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli samir-nasibli marked this pull request as ready for review July 31, 2023 23:32
@samir-nasibli
Copy link
Contributor Author

Job: http://intel-ci.intel.com/ee2ffcc8-9bec-f1c1-a4e7-a4bf010d0e2e. Specifically for sklearnex/daal4py tests/examples on GPU without dpnp on env.

@samir-nasibli
Copy link
Contributor Author

Job: http://intel-ci.intel.com/ee2ffd93-f8c6-f1de-bd22-a4bf010d0e2e. Specifically for sklearnex/daal4py tests/examples on GPU with dpctl and dpnp.

@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli samir-nasibli marked this pull request as draft August 2, 2023 09:07
@samir-nasibli samir-nasibli marked this pull request as ready for review August 2, 2023 21:57
@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli
Copy link
Contributor Author

Job: http://intel-ci.intel.com/ee318ac9-4082-f1b8-a4e7-a4bf010d0e2e. Specifically for sklearnex/daal4py tests/examples on GPU with dpctl and dpnp.

@samir-nasibli
Copy link
Contributor Author

samir-nasibli commented Aug 3, 2023

Job: http://intel-ci.intel.com/ee31f782-83c1-f190-9df2-a4bf010d0e2e. Specifically for sklearnex/daal4py tests/examples on GPU with dpctl and dpnp.

@samir-nasibli
Copy link
Contributor Author

Job: http://intel-ci.intel.com/ee321570-cbcb-f141-9c8d-a4bf010d0e2e. Specifically for sklearnex/daal4py tests/examples on GPU with dpctl and dpnp.
Using onedal with ee321570-cbcb-f141-9c8d-a4bf010d0e2e

@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli
Copy link
Contributor Author

Job: http://intel-ci.intel.com/ee35e825-3d41-f126-b1a7-a4bf010d0e2e. Specifically for sklearnex/daal4py tests/examples on GPU with dpctl and dpnp.

@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli
Copy link
Contributor Author

Job: http://intel-ci.intel.com/ee369d0a-5daa-f1ea-b9eb-a4bf010d0e2e. Specifically for sklearnex/daal4py tests/examples on GPU with dpctl and dpnp.

@samir-nasibli samir-nasibli marked this pull request as draft August 10, 2023 10:57
@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli samir-nasibli marked this pull request as ready for review August 10, 2023 14:08
@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli
Copy link
Contributor Author

@samir-nasibli
Copy link
Contributor Author

/intelci: run

@samir-nasibli
Copy link
Contributor Author

Only unnecessary check was failed and it is not related with the changes in PR at all

@samir-nasibli samir-nasibli merged commit 02465a0 into intel:master Aug 14, 2023
20 checks passed
@samir-nasibli samir-nasibli deleted the enh/dnpn_dpctl_testing branch August 14, 2023 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants