Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oneDAL development env docker image #2513

Merged
merged 50 commits into from
Sep 28, 2023

Conversation

napetrov
Copy link
Contributor

@napetrov napetrov commented Sep 13, 2023

Description

This PR introduce dev env docker image to repo

Changes proposed in this pull request:

  • docker image itself
  • validation via github actions ci for building container
  • validation via github actions Nightly for building container and ruining oneDAL build in it
  • simple readme for docker image
  • moving apt commands to separate env script
  • remove duplication for dpcpp compiler installation and move this to single script
  • create separate script for bazelisk installation and fixing bug in version management for hashing
  • moving conda install to env.yml
  • updating doc on installation with link to docker

TODO for next docker related PRs:

  • add doc build dependencies to image
  • add docbuild validation
  • add openBLAS based build validation
  • add DPC++ build validation

INSTALL.md Outdated Show resolved Hide resolved
INSTALL.md Outdated Show resolved Hide resolved
INSTALL.md Outdated Show resolved Hide resolved
dev/bazel/README.md Outdated Show resolved Hide resolved
dev/docker/README.md Outdated Show resolved Hide resolved
dev/docker/README.md Outdated Show resolved Hide resolved
dev/docker/README.md Outdated Show resolved Hide resolved
dev/docker/onedal-dev.Dockerfile Outdated Show resolved Hide resolved
dev/docker/onedal-dev.Dockerfile Outdated Show resolved Hide resolved
dev/docker/onedal-dev.Dockerfile Outdated Show resolved Hide resolved
Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>
Copy link
Contributor

@aepanchi aepanchi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

.ci/env/apt.sh Show resolved Hide resolved
dev/docker/onedal-dev.Dockerfile Show resolved Hide resolved
.ci/pipeline/ci.yml Outdated Show resolved Hide resolved
.ci/pipeline/ci.yml Outdated Show resolved Hide resolved
dev/bazel/README.md Outdated Show resolved Hide resolved
INSTALL.md Outdated Show resolved Hide resolved
Co-authored-by: Alexander Andreev <alexander.andreev@intel.com>
@napetrov napetrov merged commit 6eb63c1 into oneapi-src:master Sep 28, 2023
14 checks passed
avolkov-intel pushed a commit to avolkov-intel/oneDAL that referenced this pull request Oct 9, 2023
* removing JDK deps

* Fixing env and introduce docker

* Creating bazel env script

* adding readme for docker

* updating

* adding github validation

* new line

* remove bash invocation for scripts

* fixing ci.yml

* add execute permissions

* fixing ci.yml

* fixing github flow name

* adding sudo to docker image

* fixing install dpcpp script

* fixing base docker deps

* fixing bazelisk

* fixing ci.yml

* fixing windows ci with bash call

* fixing bazel ci

* updating bazel readme to current state

* fixing bazel caching and nuget windows build

* adding git binary

* adding build validation and conda to env

* enable larger validation in CI

* fixing CLI

* adding conda to docker

* fixing docker

* fixing docker

* adding deps and build

* adding docs env and more tests for nightly

* not passing param

* switching to conda env file

* adding conda env

* fixign path to env.yml

* drop caching version to 1 as bazel version was fixed

* bump version for spxinks-prompt to 1.8

* revert version

* disabling doc reqs

* switchign to latest tag

* actually running code in docker

* merging docker commands in one

* merging docker commands in one

* merging docker commands in one

* installing bazel in docker image

* adding build with OpenBLAS to validation

* removing build with OpenBLAS to validation

* Apply suggestions from code review

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

* Apply suggestions from code review

Co-authored-by: Alexander Andreev <alexander.andreev@intel.com>

---------

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>
Co-authored-by: Alexander Andreev <alexander.andreev@intel.com>
avolkov-intel added a commit that referenced this pull request Nov 8, 2023
* Initial commit

* Add tests

* Add cg_solver primitive to solve equation Ax = b

* Move newton_cg primitve to optimizators primitive

* Define newton_cg optimization function

* Add backtracking algorithm for optimal alpha, implement newton_cg solver

* Fix errors, add tests for newton-cg

* Remove redundant wait_and_throw, add links to sources

* Ensure code stability and fix minor issues

- Add control over the number of iterations in while loops
- Use l2-norm for convergence checks in cg-solver
- Move QuadraticFunction to primitives section

* Add sycl::fill, sycl::fabs and add specifiers for virtual functions

* Remove redundant package dependency, update default values for Float parameters

* Change update_x return type to event_vector, rename test function and minor fix

* Initial commit

* Split logloss and derivative functions, decrease the number of parameters

* Delete redundant compute functions, deselect tests

* Add LogLossFunction class and cover it with tests

* Fix bugs, rename kernels and remove redundant, update perforamnce tests

* Add wait and throw after gemv events

* Minor

* Fix error and add batch test

* Add const qualifier for table with data

* Minor

* Add LogReg test to newton_cg optimizer

* Add LogReg algorithm interface

* Add BUILD files, vanila implementation and tmp test

* Add test

* Minor

* Update train kernel

* Add GPU infer kernels to LogReg algo

* Add logistic regression dpc example

* Fix GPU test failures: PCA (#2531)

* Fix PCA tests

* Apply clang-format

* Support building reference backend with clang (#2529)

* Add correct compile flags for clang reference build

The clang makefile doesn't have the right includes defined for the
reference backend implementation. Update this to match the defines for
GCC.

* Use right include directory for TBB

The makefile doesn't use the most up-to-date variable for finding TBB
libraries. This meant that a user-installed TBB, specified through
TBBROOT, was not picked up during the build process. Now, if there is no
TBB in the __deps folder, it is picked up from TBBROOT.

* oneDAL development env docker image (#2513)

* removing JDK deps

* Fixing env and introduce docker

* Creating bazel env script

* adding readme for docker

* updating

* adding github validation

* new line

* remove bash invocation for scripts

* fixing ci.yml

* add execute permissions

* fixing ci.yml

* fixing github flow name

* adding sudo to docker image

* fixing install dpcpp script

* fixing base docker deps

* fixing bazelisk

* fixing ci.yml

* fixing windows ci with bash call

* fixing bazel ci

* updating bazel readme to current state

* fixing bazel caching and nuget windows build

* adding git binary

* adding build validation and conda to env

* enable larger validation in CI

* fixing CLI

* adding conda to docker

* fixing docker

* fixing docker

* adding deps and build

* adding docs env and more tests for nightly

* not passing param

* switching to conda env file

* adding conda env

* fixign path to env.yml

* drop caching version to 1 as bazel version was fixed

* bump version for spxinks-prompt to 1.8

* revert version

* disabling doc reqs

* switchign to latest tag

* actually running code in docker

* merging docker commands in one

* merging docker commands in one

* merging docker commands in one

* installing bazel in docker image

* adding build with OpenBLAS to validation

* removing build with OpenBLAS to validation

* Apply suggestions from code review

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

* Apply suggestions from code review

Co-authored-by: Alexander Andreev <alexander.andreev@intel.com>

---------

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>
Co-authored-by: Alexander Andreev <alexander.andreev@intel.com>

* chore(deps): update actions/checkout digest to 8ade135 (#2521)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update version to 2024.0.1 (#2533)

* Online interfaces for LOM (#2483)

* chore(deps): update dependency urllib3 to v2.0.6 [security] (#2536)

* Minor

* Change task name to binary classification and update an example

* Add template optimizer parameter to descriptor, add optimizer factory, change method to dense_batch

* Change unimplemented error name thrown in cpu kernel

* Move data to gpu before training

* Minor

* Change dependency graph to address error, fix tests

* Add optimizer descriptor as a template parameter

* Update interface, improve primitive stability
-Add l1_coef, class_count getters/setters
-Throw exception if l1_coef != 0.0 or class_count != 2
-Update example
-Add wait_and_throw for gemv events in logloss primitive to avoid
sporadic errors

* Move the optimizer dispatching logic to minimize function

* Minor

* Remove debug output, change optimizer_impl minimize interface

* Remove redundant header and fix example

* Remove redundatn imports

* Update makefile, remove dependency from rng_primitive

* Clang format

* Update makefile

* Add iterations number parameter

* Update devices list for logreg example

* Change fp type to float

* Address comments

* Adress changes, change iterations_number to iterations_count

* Substitute l1, l2 coefs to C - inverse regularization in the interface

* Remove class_count from descriptor constructor

* Fix typo in result option naming

* Add skipif statement to ignore cpu testing

* Remove redundant includes

---------

Co-authored-by: Dmitriy Ovchinnikov <117165790+inteldimitrius@users.noreply.github.com>
Co-authored-by: Keeran Rothenfusser <141222236+keeranroth@users.noreply.github.com>
Co-authored-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>
Co-authored-by: Alexander Andreev <alexander.andreev@intel.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Aleksandr Solovev <aleksandr.solovev@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants