Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add some CPU collectives to the NCCL TL #570

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Add some CPU collectives to the NCCL TL #570

wants to merge 3 commits into from

Commits on Jul 20, 2022

  1. Rename scratch_buf to barrier_scratch 

    We are going to create additional scratch buffers for other
collectives.  So instead of a generic name, such as "scratch_buf",
rename the existing buffer to "barrier_scratch", so it's more clear
what it is for.
    @pavanbalaji
    pavanbalaji committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    81c94a0 View commit details
    Browse the repository at this point in the history

  2. UCC/NCCL support for CPU buffers 

    Currently, UCC/NCCL does not support CPU buffers.  Any communication
with CPU buffers would need to go through the UCC/UCP backend, thus
requiring support for multiple backends for coverage on a single
platform.

This patch adds support for bcast and allgatherv (only the p2p
algorithm of allgatherv).
    @pavanbalaji
    pavanbalaji committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    0dc195a View commit details
    Browse the repository at this point in the history

  3. Added memory pools support to NCCL CPU collectives 

    This patch adds support to use fixed buffer length memory pools for
CPU collectives in NCCL.  This allows us to reduce the amount of
memory usage and restricts the peak memory used to scale with the
number of concurrent CPU collectives, rather than the size of the
collectives themselves.
    @pavanbalaji
    pavanbalaji committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    cfba6a4 View commit details
    Browse the repository at this point in the history