Arkouda-NJIT

This is an external repository to build functionality for Arkouda with a focus on advanced graph processing. It is built with the same structure as arkouda-contrib to manage modules and easily swap between the production (arachne) and development (arachne_development) directories.

Installing Prerequisites

We recommend following the installation instructions provided by the Arkouda development team. Most specifically, follow the Prerequisites section in its entirety, and only the Dependency Configuration section of the build instructions. The installation steps usually involve the following:

1. Download Arkouda. Use Arkouda version v2024.06.21. A specified version can be selected for download by clicking on Releases in the main repository for Arkouda.
2. Install dependencies with Anaconda. An environment containing all dependencies can be installed from arkouda-env.yml or arkouda-env-dev.yml within your Arkouda home directory.
3. Download and build Chapel. Use Chapel version 2.1.0.
4. Configure your dependencies. This involves creating (or modifying) the Makefile.paths within your Arkouda home directory.

Building the Arkouda Server with Arachne

Installation is performed through running the module_configuration.py file. The complete path to the location of arkouda must be specified through ak_loc and the complete path to the location of arachne should be specified through pkg_path.

# Print usage instructions.
python module_configuration.py --help

# Command to execute to build the Arkouda server with Arachne.
python module_configuration.py --ak_loc=/complete/path/to/arkouda/ --pkg_path=/complete/path/to/arkouda-njit/arachne/ | bash

The above command will pipe the following three commands to terminal that installs Arachne using pip, copies the Arkouda server modules to a temporary file, and combines them with the Arachne server modules to build the enhanced arkouda_server.

pip install -U /complete/path/to/arkouda-njit/arachne/client
cp /complete/path/to/arkouda/ServerModules.cfg ~/TmpServerModules.cfg.1683320760
ARKOUDA_SERVER_USER_MODULES=" /complete/path/to/arkouda-njit/arachne/server/BuildGraphMsg.chpl  /complete/path/to/arkouda-njit/arachne/server/PropertyGraphMsg.chpl  /complete/path/to/arkouda-njit/arachne/server/GraphInfoMsg.chpl  /complete/path/to/arkouda-njit/arachne/server/BFSMsg.chpl  /complete/path/to/arkouda-njit/arachne/server/TriCtrMsg.chpl  /complete/path/to/arkouda-njit/arachne/server/TriCntMsg.chpl  /complete/path/to/arkouda-njit/arachne/server/TrussMsg.chpl  /complete/path/to/arkouda-njit/arachne/server/CCMsg.chpl" ARKOUDA_CONFIG_FILE=~/TmpServerModules.cfg.1683320760 ARKOUDA_SKIP_CHECK_DEPS=true make -C /Users/alvaradoo/Research/arkouda

The server can be started as specified in the Arkouda documentation. To run a simple test file as well as pytests please proceed to the arachne folder for those instructions.

Installing Development Arachne

If you are interested in installing the development version of Arachne, please follow the same instructions as above, but for pkg_path include /complete/path/to/arkouda-njit/arachne_development/.

Usage Notes

import arkouda as ak
import arachne as ar
# code using arachne and arkouda below

Common Issues

• Issue: Unrecognized HDF5, Apache Arrow, etc. installations. Fix: Ensure Makefile.paths was properly added to the base Arkouda directory. More information can be found in the Arkouda build instructions.
• Issue: Arkouda or Arachne functions are not recognized when executing scripts. Fix: Make sure to run pip3 install -e . at both /complete/path/to/arkouda-njit/arachne/client/. and /complete/path/to/arkouda/.