Creating a ucc_context from an existing ucp_context

[nirandaperera]

hi, we have been using ucx and we create a ucp_context for our framework. Is it possible for us to create a ucc_context out of this ucp_context?

[vspetrov]

Hi, currently it is not possible.

[nirandaperera]

@vspetrov I feel like this could be an important feature.
Or else, can we get the ucp_context from the ucc_context?

[vspetrov]

There was long discussion about it. It could be nice feature indeed potentially, however it turns out to be quite complicated.
Firstly, we would need to somehow expose ucp_context/worker as part of UCC API (ucc.h). This is already a non-obvious problem, because UCC API does not anything about underlying implementation - ucp might not even be used during compilation so TL/UCP is not there.
Secondly, if you expose ucp_worker outside you have to deal with "resource sharing". E.g., two owners of ucp_worker might want to use "tagged" interface and then there is high chance of tag collision unless there is some common interface in UCP to resolve that. Same applies, e.g., for active message IDs.

Lastly, do you have any data that would show what is really the benefit ? Users will still have to allocate different sets of endpoints. What is optimized? SRQ/CQ on the worker and some mem pool objects?