Skip to content

Commit

Permalink
print sccache stats in builds (#524)
Browse files Browse the repository at this point in the history
Contributes to rapidsai/build-planning#111

Proposes some small packaging/CI changes, matching similar changes being made across RAPIDS.

* printing `sccache` stats to CI logs
* updating to the latest `rapids-dependency-file-generator` (v1.16.0)
* always explicitly specifying `cpp` / `python` in calls to `rapids-upload-wheels-to-s3`

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

URL: #524
  • Loading branch information
jameslamb authored Oct 29, 2024
1 parent f9e3466 commit 10bc842
Show file tree
Hide file tree
Showing 5 changed files with 20 additions and 4 deletions.
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ repos:
meta[.]yaml$
- id: verify-alpha-spec
- repo: https://github.com/rapidsai/dependency-file-generator
rev: v1.13.11
rev: v1.16.0
hooks:
- id: rapids-dependency-file-generator
args: ["--clean"]
Expand Down
4 changes: 4 additions & 0 deletions ci/build_cpp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ rapids-print-env
rapids-logger "Begin cpp build"
conda config --set path_conflict prevent

sccache --zero-stats

rapids-conda-retry mambabuild conda/recipes/libkvikio

sccache --show-adv-stats

rapids-upload-conda-to-s3 cpp
4 changes: 4 additions & 0 deletions ci/build_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,12 @@ rapids-logger "Begin py build"
CPP_CHANNEL=$(rapids-download-conda-from-s3 cpp)
conda config --set path_conflict prevent

sccache --zero-stats

rapids-conda-retry mambabuild \
--channel "${CPP_CHANNEL}" \
conda/recipes/kvikio

sccache --show-adv-stats

rapids-upload-conda-to-s3 python
6 changes: 5 additions & 1 deletion ci/build_wheel_cpp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,12 @@ rapids-generate-version > ./VERSION

cd "${package_dir}"

sccache --zero-stats

python -m pip install wheel
python -m pip wheel . -w dist -vvv --no-deps --disable-pip-version-check
python -m pip wheel . -w dist -v --no-deps --disable-pip-version-check

sccache --show-adv-stats

RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"
RAPIDS_PY_WHEEL_NAME="${package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 cpp dist
8 changes: 6 additions & 2 deletions ci/build_wheel_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,18 @@ cd "${package_dir}"
# are used when creating the isolated build environment
echo "libkvikio-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo ${CPP_WHEELHOUSE}/libkvikio_*.whl)" > ./constraints.txt

sccache --zero-stats

PIP_CONSTRAINT="${PWD}/constraints.txt" \
SKBUILD_CMAKE_ARGS="-DUSE_NVCOMP_RUNTIME_WHEEL=ON" \
python -m pip wheel . -w dist -vvv --no-deps --disable-pip-version-check
python -m pip wheel . -w dist -v --no-deps --disable-pip-version-check

sccache --show-adv-stats

mkdir -p final_dist
python -m auditwheel repair \
--exclude libnvcomp.so.4 \
-w final_dist \
dist/*

RAPIDS_PY_WHEEL_NAME="${package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 final_dist
RAPIDS_PY_WHEEL_NAME="${package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 python final_dist

0 comments on commit 10bc842

Please sign in to comment.