From a4be8045e553359a52c395ae3b11c92d8f64100a Mon Sep 17 00:00:00 2001 From: tess <48131946+stress-tess@users.noreply.github.com> Date: Thu, 1 Aug 2024 17:18:09 -0400 Subject: [PATCH] Closes #3508: Update installation docs (#3594) * Closes #3508: Update installation docs This PR (closes #3508) updates the installation docs. This mostly just increases version numbers. But in order to make the quickstart work I needed to add some dependencies to the arkouda user yaml and add more instructions. I also: - made the quickstart link to the latest tag rather than needing us to manually update it each release - provided a one liner for adding your conda environment to `Makefile.paths` that doesn't require the person installing to search for the right path. This will hopefully make the common case easier * rm conda install block that doesnt make sense * rm links to ENV_VARs since the relevant ones are called out during the install --------- Co-authored-by: Tess Hayes --- arkouda-env-dev.yml | 8 +++----- arkouda-env.yml | 4 +++- pydoc/quickstart.rst | 33 ++++++++++++++++++-------------- pydoc/requirements.txt | 21 +++++++++++--------- pydoc/setup/BUILD.md | 37 ++++++++++++++++++------------------ pydoc/setup/LINUX_INSTALL.md | 19 +++++++----------- pydoc/setup/MAC_INSTALL.md | 13 ++++++------- pydoc/setup/REQUIREMENTS.md | 29 +++++++++++++++++++++------- 8 files changed, 90 insertions(+), 74 deletions(-) diff --git a/arkouda-env-dev.yml b/arkouda-env-dev.yml index 1288d8dfd6..cca4ce2884 100644 --- a/arkouda-env-dev.yml +++ b/arkouda-env-dev.yml @@ -21,11 +21,11 @@ dependencies: - libidn2 - jupyter - scipy<=1.13.1 + - pytest>=6.0 + - pytest-env # Developer dependencies - pexpect - - pytest>=6.0 - - pytest-env - Sphinx>=5.1.1 - sphinx-argparse - sphinx-autoapi @@ -39,16 +39,14 @@ dependencies: - pytest-json-report - pytest-benchmark - mathjax - - sphinx-design - - sphinx-autodoc-typehints - pandas-stubs - types-python-dateutil - pip: # Developer dependencies - typeguard==2.10.0 + - sphinx-autopackagesummary - furo # sphinx theme - myst-parser - linkify-it-py - - sphinx-autopackagesummary diff --git a/arkouda-env.yml b/arkouda-env.yml index 3560d48ed1..ea4a2fe569 100644 --- a/arkouda-env.yml +++ b/arkouda-env.yml @@ -21,6 +21,8 @@ dependencies: - libidn2 - jupyter - scipy<=1.13.1 - + - pytest>=6.0 + - pytest-env + - pip: - typeguard==2.10.0 diff --git a/pydoc/quickstart.rst b/pydoc/quickstart.rst index 6754f4793e..5b3db8cd00 100644 --- a/pydoc/quickstart.rst +++ b/pydoc/quickstart.rst @@ -12,40 +12,48 @@ Install Dependencies 1. Follow the `Chapel Quickstart Guide `_ 2. Follow the `Anaconda Installation Guide `_ -3. Configure your conda environment - - .. substitution-code-block:: bash - - conda env create -f arkouda-env.yml ********************** Install Arkouda ********************** -1. Download Arkouda `v2023.05.05 `_ +1. Download either zip file from assets section of the `latest Arkouda release `_ 2. Unpack the source files: .. substitution-code-block:: bash - tar xzf arkouda-2023.05.05.tar.gz + tar xzf arkouda-RELEASE_DATE.tar.gz 3. Change to the arkouda directory .. substitution-code-block:: bash - cd arkouda-2023.05.05 + cd arkouda-RELEASE_DATE + +4. Create a conda env from `arkouda-env.yml` and activate it + + .. substitution-code-block:: bash + + conda env create -f arkouda-env.yml + conda activate arkouda -4. Build Arkouda +5. Add your conda env to `Makefile.paths`, so `make` knows where to find dependencies + + .. substitution-code-block:: bash + + echo -e "\$(eval \$(call add-path,$CONDA_PREFIX))" >> Makefile.paths + +6. Build Arkouda .. substitution-code-block:: bash make -5. Test Arkouda functionality +7. Test Arkouda functionality .. substitution-code-block:: bash - make test + make test-proto ********************** Launching the Server @@ -94,9 +102,6 @@ In another terminal window, launch an interactive Python 3 session, such as ``ip Substituting the hostname and port appropriately (defaults are 'localhost' and 5555). -When working on machines running macOS, the code block above may cause the program to hang. -This problem is resolved by running the command without the optional argument: ``ak.connect()``. - ****************************** Shutdown/Disconnect ****************************** diff --git a/pydoc/requirements.txt b/pydoc/requirements.txt index 1060a21ea0..98d80b3a93 100644 --- a/pydoc/requirements.txt +++ b/pydoc/requirements.txt @@ -12,30 +12,33 @@ h5py>=3.7.0 hdf5>=1.12.2 pip types-tabulate -tables>=3.8.0 +pytables>=3.8.0 pyarrow libiconv libidn2 +jupyter scipy<=1.13.1 +pytest>=6.0 +pytest-env # Developer dependencies pexpect -pytest>=6.0 -pytest-env Sphinx>=5.1.1 sphinx-argparse sphinx-autoapi -myst-parser +sphinx-design +sphinx-autodoc-typehints +sphinx-autopackagesummary furo # sphinx theme +myst-parser linkify-it-py typed-ast -mypy>=0.931,<0.990 flake8 +mypy>=0.931,<0.990 +black +isort +pytest-json-report pytest-benchmark mathjax -sphinx-autopackagesummary -sphinx-design -sphinx-autodoc-typehints pandas-stubs types-python-dateutil -ipython diff --git a/pydoc/setup/BUILD.md b/pydoc/setup/BUILD.md index d644b3c0b5..a9dfa863ec 100644 --- a/pydoc/setup/BUILD.md +++ b/pydoc/setup/BUILD.md @@ -4,11 +4,6 @@ If you have not yet installed the Arkouda client and prerequisites, please follow the directions in the [Installation Section](install_menu.rst) before proceeding with this build. -## Environment Variables - -In order to build the server executable, some environment variables need to be configured. -For a full list, please refer to [Environment Section](../ENVIRONMENT.md). - ## Dependency Configuration Dependencies can be configured with a package manager like `Anaconda` or manually. @@ -16,13 +11,23 @@ Dependencies can be configured with a package manager like `Anaconda` or manuall ### Using Environment Installed Dependencies *(Recommended)* When utilizing a package manager, like `Anaconda`, to install dependencies (see guides for [Linux](LINUX_INSTALL.md) or [Mac](MAC_INSTALL.md)), you will need to provide -the path to the location of your installed packages. This is achieved by adding this path to your `Makefile.paths` (Example Below). - -You might need create a `Makefile.paths` file in the top level of the arkouda directly if it doesn't already exist. +the path to the location of your installed packages. This is achieved by adding this path to your `Makefile.paths`. -It is important to note that in most cases you will only provide a single path for your environment. +In most cases you will only provide a single path for your environment. However, if you have manually installed dependencies (such as ZeroMQ or HDF5), you will need to provide each install location. +#### Using `conda` + +If you are using a conda environment to manage your dependencies. All you need to do is `conda activate` that +environment and run this command: +```commandline +echo -e "\$(eval \$(call add-path,$CONDA_PREFIX))" >> Makefile.paths +``` + +#### Using `pip` + +You might need create a `Makefile.paths` file in the top level of the arkouda directly if it doesn't already exist. + ```make # Makefile.paths @@ -31,19 +36,13 @@ $(eval $(call add-path,/home/user/anaconda3/envs/arkouda)) # ^ Note: No space after comma. ``` -The path may vary based on the installation location of Anaconda or pip and your environment name. Here are some tips to locate the path. +The path may vary based on the installation location of pip and your environment name. +Here are some tips to locate the path. ```commandline -# when installing via pip +# when installing via pip, run `pip show` on a package you've installed with pip % pip show hdf5 | grep Location -Location: /opt/homebrew/Caskroom/miniforge/base/envs/ak-base/lib/python3.10/site-packages - -# when using conda -% conda env list -# conda environments: -# -base /opt/homebrew/Caskroom/miniforge/base -ak-base * /opt/homebrew/Caskroom/miniforge/base/envs/ak-base +Location: /opt/homebrew/Caskroom/miniforge/base/envs/arkouda/lib/python3.12/site-packages ``` The `chpl` compiler will be executed with `-I`, `-L` and `-rpath` for each path in your `Makefile.paths` diff --git a/pydoc/setup/LINUX_INSTALL.md b/pydoc/setup/LINUX_INSTALL.md index d642edd310..fd44bff127 100644 --- a/pydoc/setup/LINUX_INSTALL.md +++ b/pydoc/setup/LINUX_INSTALL.md @@ -1,12 +1,5 @@ # Linux -## Update Kernel -```bash -# Update Linux kernel and install some Chapel dependencies -sudo apt-get update -sudo apt-get install gcc g++ m4 perl python3 python3-pip python3-venv python3-dev bash make mawk git pkg-config cmake llvm-14-dev llvm-14 llvm-14-tools clang-14 libclang-14-dev libclang-cpp14-dev libedit-dev -``` - ## Clone Arkouda Repository Download, clone, or fork the [arkouda repo](https://github.com/Bears-R-Us/arkouda). @@ -52,10 +45,12 @@ export PYTHONPATH="${PYTHONPATH}:${PWD}" ``` ## Chapel Installation +The first step is follow the instructions found [here](https://chapel-lang.org/docs/usingchapel/prereqs.html) +to install the Chapel Prerequisites. + For convenience, the steps to install Chapel from source are detailed below. If you need more information, please visit the -[Chapel Quickstart Guide](https://chapel-lang.org/docs/usingchapel/QUICKSTART.html) and [Chapel Prerequisites](https://chapel-lang.org/docs/usingchapel/prereqs.html). +[Chapel Quickstart Guide](https://chapel-lang.org/docs/usingchapel/QUICKSTART.html). -The first two steps in the Linux Arkouda install are to install the Chapel dependencies followed by downloading and building Chapel. The Ubuntu and RHEL Chapel installations are different for installing Chapel dependencies, particularly regarding older RHEL distro versions. Specifically, the gcc compiler on RHEL distros such as CentOS 7 do not support building Chapel. Consequently, a newer version of the gcc compiler must be installed via the @@ -67,9 +62,9 @@ be installed to enable the newer version of gcc to be leveraged for building Cha ```bash # Download latest Chapel release, explode archive, and navigate to source root directory -wget https://github.com/chapel-lang/chapel/releases/download/1.32.0/chapel-1.32.0.tar.gz -tar xvf chapel-1.32.0.tar.gz -cd chapel-1.32.0/ +wget https://github.com/chapel-lang/chapel/releases/download/2.1.0/chapel-2.1.0.tar.gz +tar xvf chapel-2.1.0.tar.gz +cd chapel-2.1.0/ # Set CHPL_HOME export CHPL_HOME=$PWD diff --git a/pydoc/setup/MAC_INSTALL.md b/pydoc/setup/MAC_INSTALL.md index f7cda13f36..83434557e6 100644 --- a/pydoc/setup/MAC_INSTALL.md +++ b/pydoc/setup/MAC_INSTALL.md @@ -7,7 +7,7 @@ Prerequisites for Arkouda can be installed using `Homebrew` or manually. Download, clone, or fork the [arkouda repo](https://github.com/Bears-R-Us/arkouda). We encourage developers to fork the repo if they expect to make any changes to arkouda. -They can then their fork and add the Bears-R-Us repo as a remote: +They can then clone their fork and add the Bears-R-Us repo as a remote: ```bash git clone https://github.com/YOUR_FORK/arkouda.git git remote add upstream https://github.com/Bears-R-Us/arkouda.git @@ -88,13 +88,13 @@ If you need more information, please visit the [Chapel Quickstart Guide](https:/ **Step 3** > Unpack the release > ```bash -> tar xzf chapel-1.32.0.tar.gz +> tar xzf chapel-2.1.0.tar.gz > ``` **Step 4** > Access the directory created when the release was unpacked > ```bash -> cd chapel-1.32.0 +> cd chapel-2.1.0 > ``` **Step 5** @@ -106,7 +106,7 @@ If you need more information, please visit the [Chapel Quickstart Guide](https:/ **Step 6** > Update environment variables to the recommended settings. > ```bash -> brew install llvm@15 +> brew install llvm > export CHPL_LLVM=system > > brew install gmp @@ -128,7 +128,7 @@ If you need more information, please visit the [Chapel Quickstart Guide](https:/ > Add the following to your `rc` file. > ```bash > # update paths to reflect where chapel and arkouda live on your machine -> export CHPL_HOME=/Users/USER/PATH_TO_CHPL/chapel-1.32.0 +> export CHPL_HOME=/Users/USER/PATH_TO_CHPL/chapel-2.1.0 > # your binary might differ especially if you have a different chipset > export PATH=$PATH:$CHPL_HOME/bin/darwin-arm64 > source ${CHPL_HOME}/util/setchplenv.bash @@ -171,8 +171,7 @@ brew install chapel ## Next Steps -We've installed Arkouda and its dependencies and built chapel with reasonable defautl environment variables. -For more information on the options, a complete list of environment variables can be found at [ENVIRONMENT.md](ENVIRONMENT.md). +We've installed Arkouda and its dependencies and built chapel with reasonable default environment variables. Now you are ready to build the server! Follow the build instructions at [BUILD.md](BUILD.md). diff --git a/pydoc/setup/REQUIREMENTS.md b/pydoc/setup/REQUIREMENTS.md index 2414f7cae7..a902f620cf 100644 --- a/pydoc/setup/REQUIREMENTS.md +++ b/pydoc/setup/REQUIREMENTS.md @@ -4,7 +4,7 @@ The installation instructions for the dependencies listed here may vary depending on your preferred operating system. Refer to the [Installation Section](install_menu.rst) for more information. -- ``Chapel 1.30.0 or later`` +- ``Chapel 2.1.0 or later`` - `cmake>=3.13.4` - `zeromq>=4.2.5` - `hdf5` @@ -30,26 +30,39 @@ The following python packages are required by the Arkouda client package. - `hdf5>=1.12.2` - `pip` - `types-tabulate` -- `tables>=3.8.0` +- `pytables>=3.8.0` - `pyarrow` +- `libiconv` +- `libidn2` +- `jupyter` - `scipy<=1.13.1` +- `pytest>=6.0` +- `pytest-env` ### Developer Specific The dependencies listed here are only required if you will be doing development for Arkouda. - `pexpect` -- `pytest>=6.0` -- `pytest-env` - `Sphinx>=5.1.1` - `sphinx-argparse` - `sphinx-autoapi` +- `sphinx-design` +- `sphinx-autodoc-typehints` +- `sphinx-autopackagesummary` - `furo` - `myst-parser` - `linkify-it-py` - `typed-ast` -- `mypy>=0.931,<0.990` - `flake8` +- `mypy>=0.931,<0.990` +- `black` +- `isort` +- `pytest-json-report` +- `pytest-benchmark` +- `mathjax` +- `pandas-stubs` +- `types-python-dateutil` ### Installing/Updating Python Dependencies @@ -62,7 +75,8 @@ Arkouda provides 2 files for installing dependencies, one for users and one for - Users Environment YAML: `arkouda-env.yml` - Developer Environment YAML: `arkouda-env-dev.yml` -**When running the commands below, replace `` with the name you want to give/have given your conda environment. Replace `` with the file appropriate to your interaction with Arkouda.** +**When running the commands below, replace `` with the name you want to give/have given your conda environment. +Replace `` with the file appropriate to your interaction with Arkouda.** ```commandline # Creating a new environment with dependencies installed @@ -75,7 +89,8 @@ conda env update -n -f --prune #### Using Pip -When you `pip install Arkouda`, dependencies should be installed as well. However, dependencies may change during the life-cycle of Arkouda, so here we detail how to update dependencies when using `pip` for package management. +When you `pip install Arkouda`, dependencies should be installed as well. However, dependencies +may change during the life-cycle of Arkouda, so here we detail how to update dependencies when using `pip` for package management. ```commandline # navigate to arkouda directory