From ad342f24beaf7678de7e3e5c8773d0a488e710a8 Mon Sep 17 00:00:00 2001 From: Yuichi Ishikawa Date: Fri, 26 Apr 2024 02:19:57 +0900 Subject: [PATCH] Change policy to use OpenBLAS --- _mathematics/acceleration/opencl.md | 28 ++++----- _mathematics/openblas/overviewopenblas.md | 76 ++++++++++++++++++----- _neuralnetworks/index.md | 2 +- _neuralnetworks/install.md | 47 ++++++-------- 4 files changed, 94 insertions(+), 59 deletions(-) diff --git a/_mathematics/acceleration/opencl.md b/_mathematics/acceleration/opencl.md index 291fe1c..d43170d 100644 --- a/_mathematics/acceleration/opencl.md +++ b/_mathematics/acceleration/opencl.md @@ -38,7 +38,7 @@ Requirements - PHP8.1 or PHP8.2 or PHP8.3 - interop-phpobjects/polite-math 1.0.6 or later - OpenCL 1.1 or later drivers/libraries. -- Windows 10, 11 or Linux (Ubuntu 20.04 or later) +- Windows 10, 11 or Linux (Ubuntu 20.04 or Debian 12 or later) - Rindow Math Matrix - Rindow OpenCL FFI - Rindow CLBlast FFI @@ -92,7 +92,7 @@ Platform(0) ``` -GPU/OpenCL support for Ubuntu +GPU/OpenCL support for Linux ------------------------------ Install the libraries required. @@ -102,20 +102,14 @@ Install the libraries required. + Set Rindow-Matlib to serial mode for use with PHP. ```shell -$ sudo apt install libopenblas-base liblapacke +$ sudo apt install libopenblas0-openmp liblapacke $ wget https://github.com/rindow/rindow-matlib/releases/download/X.X.X/rindow-matlib_X.X.X_amd64.deb $ sudo apt install ./rindow-matlib_X.X.X_amd64.deb -$ sudo update-alternatives --config librindowmatlib.so -There are 2 choices for the alternative librindowmatlib.so (providing /usr/lib/librindowmatlib.so). +``` - Selection Path Priority Status ------------------------------------------------------------- -* 0 /usr/lib/rindowmatlib-openmp/librindowmatlib.so 95 auto mode - 1 /usr/lib/rindowmatlib-openmp/librindowmatlib.so 95 manual mode - 2 /usr/lib/rindowmatlib-serial/librindowmatlib.so 90 manual mode +But if you are not allowed to use the openmp version of openblas, there is another way. +Click [here](/mathematics/openblas/overviewopenblas.html#setup-for-linux) for more information. -Press to keep the current choice[*], or type selection number: 2 -``` It is essential that OpenCL works properly in the Linux environment. (That's quite difficult) @@ -126,7 +120,7 @@ Install the OpenCL environment. $ sudo apt install clinfo $ sudo apt install intel-opencl-icd ``` -Ubuntu standard OpenCL drivers include: +Linux standard OpenCL drivers include: - mesa-opencl-icd - beignet-opencl-icd - intel-opencl-icd @@ -156,12 +150,18 @@ Scripts are available for easy download and installation. + Run script and create deb file + Install deb file +```shell +$ sudo apt install libclblast1 +``` + +If you are using Ubuntu 20.04, your distribution does not include clblast. +Download and install the clblast pre-built binaries. ```shell $ cp vendor/rindow/rindow-clblast-ffi/clblast-packdeb.sh . $ vi clblast-packdeb.sh CLBLASTVERSION=1.6.2 <===== change $ sh clblast-packdeb.sh -$ sudo apt install ./clblast_X.X.X-1+ubuntuXX.XX_amd64.deb +$ sudo apt install ./clblast_X.X.X-1_amd64.deb ``` Configure the rindow-math-matrix. diff --git a/_mathematics/openblas/overviewopenblas.md b/_mathematics/openblas/overviewopenblas.md index 294258e..4762445 100644 --- a/_mathematics/openblas/overviewopenblas.md +++ b/_mathematics/openblas/overviewopenblas.md @@ -26,7 +26,7 @@ requirements ------------ - PHP8.1 or PHP8.2 or PHP8.3 -- Linux or Windows 10, 11 +- Windows 10, 11 or Linux (Ubuntu 20.04 or Debian 12 or later) - OpenBLAS library 0.3.20 or later - Rindow-Math Library 1.0 or later @@ -71,7 +71,7 @@ LAPACK Driver : Rindow\OpenBLAS\FFI\Lapack Math Driver : Rindow\Matlib\FFI\Matlib ``` -### Setup for Ubuntu +### Setup for Linux Install each library using the apt command. @@ -81,25 +81,34 @@ $ php -m | grep FFI FFI ``` -Install the fast matrix calculation library. -And then set the rindow-matlib to serial mode for use with PHP. +**Install OpenBLAS:** + +Since rindow-matlib currently uses OpenMP, choose the OpenMP version for OpenBLAS as well. + +Using the pthread version of OpenBLAS can cause conflicts and become unstable and slow. +This issue does not occur on Windows. + +```shell +$ sudo apt install libopenblas0-openmp liblapacke +``` + +**Install Rindow-Matlib:** + +Download the pre-build binary file. + +- https://github.com/rindow/rindow-matlib/releases + +Please install using the apt command. ```shell -$ mkdir -p /your/project/directory -$ cd /your/project/directory -$ sudo apt install libopenblas-base liblapacke -$ wget https://github.com/rindow/rindow-matlib/releases/download/X.X.X/rindow-matlib_X.X.X_amd64.deb $ sudo apt install ./rindow-matlib_X.X.X_amd64.deb -$ sudo update-alternatives --config librindowmatlib.so -There are 2 choices for the alternative librindowmatlib.so (providing /usr/lib/librindowmatlib.so). +``` +Just it. - Selection Path Priority Status ------------------------------------------------------------- -* 0 /usr/lib/rindowmatlib-openmp/librindowmatlib.so 95 auto mode - 1 /usr/lib/rindowmatlib-openmp/librindowmatlib.so 95 manual mode - 2 /usr/lib/rindowmatlib-serial/librindowmatlib.so 90 manual mode +**Install Rindow-Matlib-FFI:** -Press to keep the current choice[*], or type selection number: 2 +Set it up using composer. +```shell $ cd \your\progject\directory $ composer require rindow/rindow-math-matrix $ composer require rindow/rindow-math-matrix-matlibffi @@ -111,3 +120,38 @@ LAPACK Driver : Rindow\OpenBLAS\FFI\Lapack Math Driver : Rindow\Matlib\FFI\Matlib ``` + +### Troubleshooting for Linux + +If you have already installed the pthread version of OpenBLAS, +```shell +$ sudo apt remove libopenblas0-pthread +``` + +But if you can't remove it, you can switch to it using the update-alternatives command. + +```shell +$ sudo update-alternatives --config libopenblas.so.0-x86_64-linux-gnu +$ sudo update-alternatives --config liblapack.so.3-x86_64-linux-gnu +``` + +If you really want to use the pthread version of OpenBLAS, please switch to the serial version of rindow-matlib. + +There are no operational mode conflicts with OpenBLAS on Windows. + +But, If you really want to use the pthread version of OpenBLAS, please switch to the serial version of rindow-matlib. + +```shell +$ sudo update-alternatives --config librindowmatlib.so +There are 2 choices for the alternative librindowmatlib.so (providing /usr/lib/librindowmatlib.so). + + Selection Path Priority Status +------------------------------------------------------------ +* 0 /usr/lib/rindowmatlib-openmp/librindowmatlib.so 95 auto mode + 1 /usr/lib/rindowmatlib-openmp/librindowmatlib.so 95 manual mode + 2 /usr/lib/rindowmatlib-serial/librindowmatlib.so 90 manual mode + +Press to keep the current choice[*], or type selection number: 2 +``` +Choose the "rindowmatlib-serial". + diff --git a/_neuralnetworks/index.md b/_neuralnetworks/index.md index f9b49f6..63a3e1e 100644 --- a/_neuralnetworks/index.md +++ b/_neuralnetworks/index.md @@ -131,7 +131,7 @@ Requirements ------------ - PHP 8.1 8.2 8.3 (If you want to use it on PHP 7.x and 8.0, please use Version 1.x.) -- Windows 10 or Ubuntu 20.04 or later is required to use OpenBLAS and Rindow-Matlib. +- Windows 10 or Ubuntu 20.04 or Debian 12 or later is required to use OpenBLAS and Rindow-Matlib. - Rindow Math Matrix Recommends diff --git a/_neuralnetworks/install.md b/_neuralnetworks/install.md index 7b28277..e75b618 100644 --- a/_neuralnetworks/install.md +++ b/_neuralnetworks/install.md @@ -8,21 +8,21 @@ next_section: tutorials/tutorials - [Operating environment](#operating-environment) - [Installation instructions for Windows](#installation-instructions-for-windows) -- [Installation instructions for Ubuntu](#installation-instructions-for-ubuntu) +- [Installation instructions for Linux](#installation-instructions-for-linux) - [GPU/OpenCL support for Windows](#gpuopencl-support-for-windows) -- [GPU/OpenCL support for Ubuntu](#gpuopencl-support-for-ubuntu) +- [GPU/OpenCL support for Linux](#gpuopencl-support-for-linux) Operating environment --------------------- Rindow Neural Networks has been tested in the following operating environments: -・PHP 8.1, 8.2, 8.3 (When using in PHP 7.x, 8.0 environment, please use Release 1.x.) +- PHP 8.1, 8.2, 8.3 (When using in PHP 7.x, 8.0 environment, please use Release 1.x.) - Windows 10 20H2 or later. -- Ubuntu 20.04, 22.04 +- Ubuntu 20.04, 22.04 or Debian 12 or later - AMD/Intel CPU/APU 64bit (SSE2 or later) -- OpenBLAS (0.3.20 Windows-x64, 0.3.20 Ubuntu-2204, 0.3.8 Ubuntu-2004) -- CLBlast (1.5.2 or later, Windows-x64, Ubuntu-2204, Ubuntu-2004) +- OpenBLAS (>=0.3.20:Windows-x64, 0.3.20:Ubuntu-2204, 0.3.8:Ubuntu-2004, 0.3.21:Debian12) +- CLBlast (>=1.5.2:Windows-x64, 1.5.2:Ubuntu-2204, >=1.5.2:Ubuntu-2004, 1.5.3:Debian12) It also works with Intel/AMD CPU/APU and integrated graphics with OpenCL drivers. @@ -137,7 +137,7 @@ Epoch 5/5 [.........................] 1 sec. remaining:00:00 - 2 sec. The graph is displayed ``` -Installation instructions for Ubuntu +Installation instructions for Linux ----------------------------------------- Install php. @@ -179,20 +179,12 @@ Install the libraries required by Rindow NeuralNetworks. + Set Rindow-Matlib to serial mode for use with PHP. ```shell -$ sudo apt install libopenblas-base liblapacke +$ sudo apt install libopenblas0-openmp liblapacke $ wget https://github.com/rindow/rindow-matlib/releases/download/X.X.X/rindow-matlib_X.X.X_amd64.deb $ sudo apt install ./rindow-matlib_X.X.X_amd64.deb -$ sudo update-alternatives --config librindowmatlib.so -There are 2 choices for the alternative librindowmatlib.so (providing /usr/lib/librindowmatlib.so). - - Selection Path Priority Status ------------------------------------------------------------- -* 0 /usr/lib/rindowmatlib-openmp/librindowmatlib.so 95 auto mode - 1 /usr/lib/rindowmatlib-openmp/librindowmatlib.so 95 manual mode - 2 /usr/lib/rindowmatlib-serial/librindowmatlib.so 90 manual mode - -Press to keep the current choice[*], or type selection number: 2 ``` +But if you are not allowed to use the openmp version of openblas, there is another way. +Click [here](/mathematics/openblas/overviewopenblas.html#troubleshooting-for-Linux) for more information. Install Rindow Neural Networks. @@ -298,7 +290,7 @@ Platform(0) ``` -GPU/OpenCL support for Ubuntu +GPU/OpenCL support for Linux ------------------------------ It is essential that OpenCL works properly in the Linux environment. (That's quite difficult) @@ -309,7 +301,7 @@ Install the OpenCL environment. $ sudo apt install clinfo $ sudo apt install intel-opencl-icd ``` -Ubuntu standard OpenCL drivers include: +Linux standard OpenCL drivers include: - mesa-opencl-icd - beignet-opencl-icd - intel-opencl-icd @@ -330,21 +322,20 @@ Number of platforms 1 .. ``` -Download and install the CLBlast library. -Scripts are available for easy download and installation. +Install the CLBlast library. -+ Check the latest version: [CLBlast library](https://github.com/CNugteren/CLBlast/releases) -+ Copy script -+ Change the version at the beginning of the script -+ Run script and create deb file -+ Install deb file +```shell +$ sudo apt install libclblast1 +``` +If you are using Ubuntu 20.04, your distribution does not include clblast. +Download and install the clblast pre-built binaries. ```shell $ cp vendor/rindow/rindow-clblast-ffi/clblast-packdeb.sh . $ vi clblast-packdeb.sh CLBLASTVERSION=1.6.2 <===== change $ sh clblast-packdeb.sh -$ sudo apt install ./clblast_X.X.X-1+ubuntuXX.XX_amd64.deb +$ sudo apt install ./clblast_X.X.X-1_amd64.deb ``` Configure the rindow-neuralnetworks backend to use OpenCL.