TL/UCP: Add linear alltoall and allgather algorithms based on xgvmi ucp get #992
Conversation
nsarka
force-pushed
the
nsarka/allgather_xgvmi
branch
from
3b8cbf2
to
ecbd9e1
Compare
nsarka
force-pushed
the
nsarka/allgather_xgvmi
branch
from
ecbd9e1
to
a362467
Compare
nsarka
changed the title
|
I don't think we should do this now, but these algorithms, including sliding-window AR will not need the allgather in the init function when #909 is merged |
|
|
||
| req_param.op_attr_mask |= UCP_OP_ATTR_FIELD_MEMH; | ||
|
|
||
| for (i = *posted; i < host_team_size; i++) { |
There was a problem hiding this comment.
is it possible that posted is not 0 when entering this function?
Would it make sense to put this first loop in a "start" function instead?
There was a problem hiding this comment.
On the first entry *posted will be 0, but after that it will be equal to host_team_size. It's possible to make a standalone start function, but I thought it would be less code to reuse ucc_tl_ucp_dpu_xgvmi_rdma_task_post for both alltoall and allgather, even though it isn't that long (10 lines)
|
|
||
| ucp_worker_progress(tl_ctx->worker.ucp_worker); | ||
|
|
||
| for (i = *completed; i < *posted; i++) { |
There was a problem hiding this comment.
here *posted is necessarily equal to host_team_size right?
There was a problem hiding this comment.
yes, when it gets to this point all the gets will be posted. I think *posted might be clearer, just because we want *posted == *completed at the end. What do you think?
|
Can you update the tests as well? |
nsarka
force-pushed
the
nsarka/allgather_xgvmi
branch
from
8e17a30
to
419a128
Compare
I updated the gtests to test the linear allgather/alltoall |
nsarka
force-pushed
the
nsarka/allgather_xgvmi
branch
from
873b576
to
ea42acf
Compare
Co-authored-by: samnordmann <snordmann@nvidia.com>
nsarka
force-pushed
the
nsarka/allgather_xgvmi
branch
from
ea42acf
to
925c3bb
Compare
This PR is a follow up to allreduce sliding window. It adds linear alltoall and allgather algorithms based on XGVMI. They will post ucp gets from host to host in a round robin fashion.