Merge pull request #378 from rapidsai/branch-22.06

[RELEASE] cuxfilter v22.06

CHANGELOG.md

@@ -1,3 +1,25 @@
+# cuXfilter 22.06.00 (7 Jun 2022)
+
+## 🔗 Links
+
+- [Development Branch](https://github.com/rapidsai/cuxfilter/tree/branch-22.06)
+- [Compare with `main` branch](https://github.com/rapidsai/cuxfilter/compare/main...branch-22.06)
+
+
+## 🐛 Bug Fixes
+
+- Fixed native support for dask_cudf dataframes. Seamless integration results in a dask_cudf.DataFrame working as a drop-in replacement for a cudf.DataFrame([#359, #366](https://github.com/rapidsai/cuxfilter/pull/359, #366)) [@AjayThorve](https://github.com/AjayThorve)
+
+## 🛠️ Improvements
+
+- added `unseleced_alpha` parameter to all datashader charts, displays unselected data as transparent (default alpha=0.2) ([#366](https://github.com/rapidsai/cuxfilter/pull/366))
+- added binary data transfer support for choropleth charts, which results in a much smoother experience interacting with the choropleth charts ([#366](https://github.com/rapidsai/cuxfilter/pull/366))
+- Simplify conda recipe ([#373](https://github.com/rapidsai/cuxfilter/pull/373)) [@Ethyling](https://github.com/Ethyling)
+- Forward-merge branch-22.04 to branch-22.06 ([#370](https://github.com/rapidsai/cuxfilter/pull/370)) [@Ethyling](https://github.com/Ethyling)
+- Use conda to build python packages during GPU tests ([#368](https://github.com/rapidsai/cuxfilter/pull/368)) [@Ethyling](https://github.com/Ethyling)
+- Use conda compilers ([#351](https://github.com/rapidsai/cuxfilter/pull/351)) [@Ethyling](https://github.com/Ethyling)
+- Build packages using mambabuild ([#347](https://github.com/rapidsai/cuxfilter/pull/347)) [@Ethyling](https://github.com/Ethyling)
+
 # cuXfilter 22.04.00 (6 Apr 2022)
 
 ## 🐛 Bug Fixes

README.md

@@ -2,15 +2,15 @@
 
 [![Build Status](https://gpuci.gpuopenanalytics.com/job/rapidsai/job/gpuci/job/cuxfilter/job/branches/job/cuxfilter-branch-pipeline/badge/icon)](https://gpuci.gpuopenanalytics.com/job/rapidsai/job/gpuci/job/cuxfilter/job/branches/job/cuxfilter-branch-pipeline/)
 
-cuxfilter ( ku-cross-filter ) is a [RAPIDS](https://github.com/rapidsai) framework to connect web visualizations to GPU accelerated crossfiltering. Inspired by the javascript version of the [original]( https://github.com/crossfilter/crossfilter), it enables interactive and super fast multi-dimensional filtering of 100 million+ row tabular datasets via [cuDF](https://github.com/rapidsai/cudf). 
-
+cuxfilter ( ku-cross-filter ) is a [RAPIDS](https://github.com/rapidsai) framework to connect web visualizations to GPU accelerated crossfiltering. Inspired by the javascript version of the [original]( https://github.com/crossfilter/crossfilter), it enables interactive and super fast multi-dimensional filtering of 100 million+ row tabular datasets via [cuDF](https://github.com/rapidsai/cudf).
 
 ## RAPIDS Viz
-cuxfilter is one of the core projects of the “RAPIDS viz” team. Taking the axiom that “a slider is worth a thousand queries” from @lmeyerov to heart, we want to enable fast exploratory data analytics through an easier-to-use pythonic notebook interface. 
+
+cuxfilter is one of the core projects of the “RAPIDS viz” team. Taking the axiom that “a slider is worth a thousand queries” from @lmeyerov to heart, we want to enable fast exploratory data analytics through an easier-to-use pythonic notebook interface.
 
 As there are many fantastic visualization libraries available for the web, our general principle is not to create our own viz library, but to enhance others with faster acceleration, larger datasets, and better dev UX. **Basically, we want to take the headache out of interconnecting multiple charts to a GPU backend, so you can get to visually exploring data faster.**
 
-By the way, cuxfilter is best used to interact with large (1 million+) tabular datasets. GPU’s are fast, but accessing that speedup requires some architecture overhead that isn’t worthwhile for small datasets. 
+By the way, cuxfilter is best used to interact with large (1 million+) tabular datasets. GPU’s are fast, but accessing that speedup requires some architecture overhead that isn’t worthwhile for small datasets.
 
 For more detailed requirements, see below.
 
@@ -22,7 +22,7 @@ The current version of cuxfilter leverages jupyter notebook and bokeh server to
 
 ### What is cuDataTiles?
 
-cuxfilter implements cuDataTiles, a GPU accelerated version of data tiles based on the work of [Falcon](https://github.com/uwdata/falcon). When starting to interact with specific charts in a cuxfilter dashboard, values for the other charts are precomputed to allow for fast slider scrubbing without having to recalculate values. 
+cuxfilter implements cuDataTiles, a GPU accelerated version of data tiles based on the work of [Falcon](https://github.com/uwdata/falcon). When starting to interact with specific charts in a cuxfilter dashboard, values for the other charts are precomputed to allow for fast slider scrubbing without having to recalculate values.
 
 ### Open Source Projects
 
@@ -34,17 +34,16 @@ cuxfilter wouldn’t be possible without using these great open source projects:
 - [Falcon](https://github.com/uwdata/falcon)
 - [Jupyter](https://jupyter.org/about)
 
-
 ### Where is the original cuxfilter and Mortgage Viz Demo?
 
 The original version (0.2) of cuxfilter, most known for the backend powering the Mortgage Viz Demo, has been moved into the [`GTC-2018-mortgage-visualization branch`](https://github.com/rapidsai/cuxfilter/tree/GTC-2018-mortgage-visualization) branch. As it has a much more complicated backend and javascript API, we’ve decided to focus more on the streamlined notebook focused version here.
 
-
 ## Usage
 
-### Example 1 
+### Example 1
+
 [![Open In Studio Lab](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github/rapidsai/cuxfilter/blob/branch-22.02/notebooks/auto_accidents_example.ipynb) [<img src="https://img.shields.io/badge/-Setup Studio Lab Environment-gray.svg">](./notebooks/README.md#amazon-sagemaker-studio-lab)
-    
+
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rapidsai/cuxfilter/blob/branch-22.02/notebooks/auto_accidents_example.ipynb) [<img src="https://img.shields.io/badge/-Setup Colab Environment-gray.svg">](./notebooks/README.md#google-colab)
 
 ```python
@@ -63,7 +62,7 @@ gtc_demo_red_blue_palette = [ "#3182bd", "#6baed6", "#7b8ed8", "#e26798", "#ff00
 
 #declare charts
 chart1 = cuxfilter.charts.scatter(x='dropoff_x', y='dropoff_y', aggregate_col='DAY_WEEK', aggregate_fn='mean',
-                                color_palette=gtc_demo_red_blue_palette, tile_provider='CARTODBPOSITRON',
+                                color_palette=gtc_demo_red_blue_palette, tile_provider='CartoLight', unselected_alpha=0.2,
                                 pixel_shade_type='linear')
 chart2 = cuxfilter.charts.multi_select('YEAR')
 chart3 = cuxfilter.charts.bar('DAY_WEEK', x_label_map=label_map)
@@ -79,11 +78,13 @@ d = cux_df.dashboard([chart1, chart3, chart4], sidebar=[chart2], layout=cuxfilte
 d.app()
 
 ```
+
 ![output dashboard](./docs/_images/demo.gif)
 
-### Example 2 
+### Example 2
+
 [![Open In Studio Lab](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github/rapidsai/cuxfilter/blob/branch-22.02/notebooks/Mortgage_example.ipynb) [<img src="https://img.shields.io/badge/-Setup Studio Lab Environment-gray.svg">](./notebooks/README.md#amazon-sagemaker-studio-lab)
-    
+
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rapidsai/cuxfilter/blob/branch-22.02/notebooks/Mortgage_example.ipynb) [<img src="https://img.shields.io/badge/-Setup Colab Environment-gray.svg">](./notebooks/README.md#google-colab)
 
 ```python
@@ -117,16 +118,15 @@ d = cux_df.dashboard([chart0, chart2],sidebar=[chart3, chart1], layout=cuxfilter
 d.show('jupyter-notebook/lab-url')
 
 ```
-![output dashboard](./docs/_images/demo2.gif)
 
+![output dashboard](./docs/_images/demo2.gif)
 
 ## Documentation
 
 Full documentation can be found [on the RAPIDS docs page](https://docs.rapids.ai/api/cuxfilter/stable/).
 
 Troubleshooting help can be found [on our troubleshooting page](https://docs.rapids.ai/api/cuxfilter/stable/installation.html#troubleshooting).
 
-
 ## General Dependencies
 
 - python
@@ -146,26 +146,27 @@ Please see the [Demo Docker Repository](https://hub.docker.com/r/rapidsai/rapids
 
 ## Installation
 
-
 ### CUDA/GPU requirements
 
-* CUDA 10.1+
-* NVIDIA driver 418.39+
-* Pascal architecture or better (Compute Capability >=6.0)
+- CUDA 10.1+
+- NVIDIA driver 418.39+
+- Pascal architecture or better (Compute Capability >=6.0)
 
 ### Conda
 
 cuxfilter can be installed with conda ([miniconda](https://conda.io/miniconda.html), or the full [Anaconda distribution](https://www.anaconda.com/download)) from the `rapidsai` channel:
 
-For `cuxfilter version == 22.04` :
+For `cuxfilter version == 22.06` :
+
 ```bash
 # for CUDA 11.5
 conda install -c rapidsai -c nvidia -c numba -c conda-forge \
-    cuxfilter=22.04 python=3.7 cudatoolkit=11.5
+    cuxfilter=22.06 python=3.7 cudatoolkit=11.5
 
 ```
 
 For the nightly version of `cuxfilter` :
+
 ```bash
 # for CUDA 11.5
 conda install -c rapidsai-nightly -c nvidia -c numba -c conda-forge \
@@ -174,13 +175,11 @@ conda install -c rapidsai-nightly -c nvidia -c numba -c conda-forge \
 
 Note: cuxfilter is supported only on Linux, and with Python versions 3.7 and later.
 
-See the [Get RAPIDS version picker](https://rapids.ai/start.html) for more OS and version info. 
-
+See the [Get RAPIDS version picker](https://rapids.ai/start.html) for more OS and version info.
 
 ### Build/Install from Source
-See [build instructions](CONTRIBUTING.md#setting-up-your-build-environment).
-
 
+See [build instructions](CONTRIBUTING.md#setting-up-your-build-environment).
 
 ## Troubleshooting
 
@@ -232,6 +231,7 @@ python -c "from cuxfilter.sampledata import datasets_check; datasets_check(base_
 Currently supported layout templates and example code can be found on the [layouts page](https://rapidsai.github.io/cuxfilter/layouts/Layouts.html).
 
 ### Currently Supported Charts
+
 | Library  | Chart type |
 | ------------- | ------------- |
 | bokeh  | bar, line  |
@@ -249,4 +249,5 @@ You can see the examples to implement viz libraries in the bokeh and cudatashade
 For more details, check out the [contributing guide](./CONTRIBUTING.md).
 
 ## Future Work
+
 cuxfilter development is in early stages and on going. See what we are planning next on the [projects page](https://github.com/rapidsai/cuxfilter/projects).

ci/cpu/build.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# COPYRIGHT (c) 2020, NVIDIA CORPORATION.
+# COPYRIGHT (c) 2020-2022, NVIDIA CORPORATION.
 ######################################
 # cuXfilter CPU conda build script for CI #
 ######################################
@@ -61,12 +61,15 @@ conda list --show-channel-urls
 # FIX Added to deal with Anancoda SSL verification issues during conda builds
 conda config --set ssl_verify False
 
+# FIXME: Remove
+gpuci_mamba_retry install -c conda-forge boa
+
 ################################################################################
 # BUILD - Conda package builds
 ################################################################################
 
 echo "Building cuxfilter"
-gpuci_conda_retry build conda/recipes/cuxfilter --python=$PYTHON
+gpuci_conda_retry mambabuild conda/recipes/cuxfilter --python=$PYTHON
 
 ################################################################################
 # UPLOAD - Conda packages

ci/gpu/build.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# COPYRIGHT (c) 2020, NVIDIA CORPORATION.
+# COPYRIGHT (c) 2020-2002, NVIDIA CORPORATION.
 ##############################################
 # cuXfilter GPU build and test script for CI #
 ##############################################
@@ -24,6 +24,8 @@ export HOME="$WORKSPACE"
 cd "$WORKSPACE"
 export GIT_DESCRIBE_TAG=`git describe --tags`
 export MINOR_VERSION=`echo $GIT_DESCRIBE_TAG | grep -o -E '([0-9]+\.[0-9]+)'`
+unset GIT_DESCRIBE_TAG
+
 # Set `LIBCUDF_KERNEL_CACHE_PATH` environment variable to $HOME/.jitify-cache because
 # it's local to the container's virtual file system, and not shared with other CI jobs
 # like `/tmp` is.
@@ -53,37 +55,31 @@ nvidia-smi
 gpuci_logger "Activate conda env"
 . /opt/conda/etc/profile.d/conda.sh
 conda activate rapids
-gpuci_mamba_retry install -y \
-               "cudf=$MINOR_VERSION.*" "cudatoolkit=$CUDA_REL" \
-               "cugraph=$MINOR_VERSION.*" \
-               "cuspatial=$MINOR_VERSION.*" \
-               "dask-cudf=$MINOR_VERSION.*" "dask-cuda=$MINOR_VERSION.*" \
-               "numba>=0.54" \
-               "bokeh>=2.4.2,<=2.5" \
-               "rapids-build-env=$MINOR_VERSION.*" \
-               "rapids-notebook-env=$MINOR_VERSION.*"
-
-# https://docs.rapids.ai/maintainers/depmgmt/ 
-# conda remove --force rapids-build-env rapids-notebook-env
-# conda install "your-pkg=1.0.0"
 
 gpuci_logger "Check versions"
 python --version
 $CC --version
 $CXX --version
+
 conda info
 conda config --show-sources
 conda list --show-channel-urls
 
 ################################################################################
-# BUILD - Build cuxfilter from source
+# BUILD - Build cuxfilter
 ################################################################################
 
-gpuci_logger "Build cuxfilter"
-"$WORKSPACE/build.sh" clean cuxfilter
+# TODO: Move boa install to gpuci/rapidsai
+gpuci_mamba_retry install boa
+
+gpuci_logger "Build and install cuxfilter"
+cd "${WORKSPACE}"
+CONDA_BLD_DIR="${WORKSPACE}/.conda-bld"
+gpuci_conda_retry mambabuild  --croot "${CONDA_BLD_DIR}" conda/recipes/cuxfilter --python=$PYTHON
+gpuci_mamba_retry install -c "${CONDA_BLD_DIR}" cuxfilter
 
 ################################################################################
-# TEST - Run pytest 
+# TEST - Run pytest
 ################################################################################
 
 set +e -Eo pipefail
@@ -93,6 +89,9 @@ trap "EXITCODE=1" ERR
 if hasArg --skip-tests; then
     gpuci_logger "Skipping Tests"
 else
+    gpuci_logger "Install tests dependencies"
+    gpuci_mamba_retry install "cugraph=${MINOR_VERSION}.*"
+
     gpuci_logger "Check GPU usage"
     nvidia-smi
 

conda/environments/cuxfilter_dev_cuda11.0.yml

@@ -8,13 +8,13 @@ channels:
 - anaconda
 dependencies:
 - cupy>7.1.0,<10.0.0a0
-- cuspatial=22.04
+- cuspatial=22.06
 - python>=3.6,<3.8
-- cudf=22.04
+- cudf=22.06
 - dask>=2.12.0
 - distributed>=2.12.0
-- dask-cuda=22.04
-- dask-cudf=22.04
+- dask-cuda=22.06
+- dask-cudf=22.06
 - datashader>0.13, <0.14
 - holoviews>1.14.1, <=1.14.6
 - numba>=0.54
@@ -39,7 +39,7 @@ dependencies:
 - ipython
 - jupyterlab
 - flake8=3.8.3
-- black=19.10
+- black=22.3.0
 - pydeck>=0.3, <=0.5.0
 - packaging
 - pip

conda/environments/cuxfilter_dev_cuda11.2.yml

@@ -8,13 +8,13 @@ channels:
 - anaconda
 dependencies:
 - cupy>7.1.0,<10.0.0a0
-- cuspatial=22.04
+- cuspatial=22.06
 - python>=3.6,<3.8
-- cudf=22.04
+- cudf=22.06
 - dask>=2.12.0
 - distributed>=2.12.0
-- dask-cuda=22.04
-- dask-cudf=22.04
+- dask-cuda=22.06
+- dask-cudf=22.06
 - datashader>0.13, <0.14
 - holoviews>1.14.1, <=1.14.6
 - numba>=0.54
@@ -39,7 +39,7 @@ dependencies:
 - ipython
 - jupyterlab
 - flake8=3.8.3
-- black=19.10
+- black=22.3.0
 - pydeck>=0.3, <=0.5.0
 - packaging
 - pip

conda/environments/cuxfilter_dev_cuda11.4.yml

@@ -8,13 +8,13 @@ channels:
 - anaconda
 dependencies:
 - cupy>7.1.0,<10.0.0a0
-- cuspatial=22.04
+- cuspatial=22.06
 - python>=3.6,<3.8
-- cudf=22.04
+- cudf=22.06
 - dask>=2.12.0
 - distributed>=2.12.0
-- dask-cuda=22.04
-- dask-cudf=22.04
+- dask-cuda=22.06
+- dask-cudf=22.06
 - datashader>0.13, <0.14
 - holoviews>1.14.1, <=1.14.6
 - numba>=0.54
@@ -39,7 +39,7 @@ dependencies:
 - ipython
 - jupyterlab
 - flake8=3.8.3
-- black=19.10
+- black=22.3.0
 - pydeck>=0.3, <=0.5.0
 - packaging
 - pip

conda/environments/cuxfilter_dev_cuda11.5.yml

@@ -8,13 +8,13 @@ channels:
 - anaconda
 dependencies:
 - cupy>7.1.0,<10.0.0a0
-- cuspatial=22.04
+- cuspatial=22.06
 - python>=3.6,<3.8
-- cudf=22.04
+- cudf=22.06
 - dask>=2.12.0
 - distributed>=2.12.0
-- dask-cuda=22.04
-- dask-cudf=22.04
+- dask-cuda=22.06
+- dask-cudf=22.06
 - datashader>0.13, <0.14
 - holoviews>1.14.1, <=1.14.6
 - numba>=0.54
@@ -39,7 +39,7 @@ dependencies:
 - ipython
 - jupyterlab
 - flake8=3.8.3
-- black=19.10
+- black=22.3.0
 - pydeck>=0.3, <=0.5.0
 - packaging
 - pip

conda/recipes/cuxfilter/conda_build_config.yaml

@@ -0,0 +1,8 @@
+c_compiler_version:
+  - 9
+
+cxx_compiler_version:
+  - 9
+
+sysroot_version:
+  - "2.17"