Skip to content
/ python-cudf Public template

For graphics processing unit accelerated data science via rapids.ai

Notifications You must be signed in to change notification settings

thetemplates/python-cudf

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

42 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation


Template for rapids.ai


Template for rapids.ai


RAPIDS

This template is for developing rapids.ai dependent products; rapids.ai enables graphics processing unit accelerated data science.


Remote Development

Via a remote development image. The image depends on .devcontainer/Dockerfile & .devcontainer/requirements.txt, and is built via

 docker build . --file .devcontainer/Dockerfile -t rapids

Run an instance of the image, i.e., a container, via

docker run --rm --gpus all -i -t -p 127.0.0.1:10000:8888 -w /app 
	--mount type=bind,src="$(pwd)",target=/app rapids

or

docker run --rm --gpus all -i -t -p 127.0.0.1:10000:8888 -w /app 
	--mount type=bind,src="$(pwd)",target=/app 
	-v ~/.aws:/home/rapids/.aws rapids

Opt for the second option if you need to interact with Amazon Web Services. Note:

  • --rm: automatically remove container
  • -i: interactive mode
  • -t: tag
  • -p: publish maps the host port 10000 to container port 8888
  • -v ~/.aws:/home/rapids/.aws enables interactions with Amazon Web Services, if such interactions are required. It maps the local Amazon Web Services credentials directory to a temporary .aws directory within the container.

Get the name of the container via:

docker ps --all

Remote Development & Integrated Development Environments

In aid of productivity, use/attach an IDE (integrated development environment) to the running container. The IntelliJ IDEA set up involves connecting to a machine's Docker daemon, the steps are


  • Settings $\rightarrow$ Build, Execution, Deployment $\rightarrow$ Docker $\rightarrow$ WSL: {select the linux operating system}
  • View $\rightarrow$ Tool Window $\rightarrow$ Services
    Within the Containers section connect to the running instance of interest, or ascertain connection to the running instance of interest.

Visual Studio Code has its container attachment instructions; study Attach Container.



Code Analysis

The GitHub Actions script main.yml conducts code analysis within a Cloud GitHub Workspace. Depending on the script, code analysis may occur on push to any repository branch, or on push to a specific branch.

The sections herein outline remote code analysis.


pylint

The directive

pylint --generate-rcfile > .pylintrc

generates the dotfile .pylintrc of the static code analyser pylint. Analyse a directory via the command

python -m pylint --rcfile .pylintrc {directory}

The .pylintrc file of this template project has been amended to adhere to team norms, including

  • Maximum number of characters on a single line.

    max-line-length=127

  • Maximum number of lines in a module.

    max-module-lines=135


pytest

The pytest framework is for test writing. This template includes a test for the program src/algorithms/example.py. The program's corresponding test program is tests/algorithms/test_example.py, which is run via

python -m pytest -p no:cacheprovider /app/tests/algorithms/test_example.py

The directive no:cacheprovider will prevent the storage of test cache, hence prevents the need for a .pytest_cache directory.


flake8

For code & complexity analysis. A directive of the form

python -m flake8 --count --select=E9,F63,F7,F82 --show-source 
	--statistics src/{directory.name}

inspects issues in relation to logic (F7), syntax (Python E9, Flake F7), mathematical formulae symbols (F63), undefined variable names (F82). Additionally

python -m flake8 --count --exit-zero --max-complexity=10 --max-line-length=127 
	--statistics src/{directory.name}

inspects complexity.



References

Images via Docker Hub Container Image Library

Images via NVIDIA GPU Cloud (NGC)

Additionally









About

For graphics processing unit accelerated data science via rapids.ai

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages