Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] send and recv use multiple cpus #131

Open
da-niao-dan opened this issue Jun 6, 2023 · 2 comments
Open

[Feature] send and recv use multiple cpus #131

da-niao-dan opened this issue Jun 6, 2023 · 2 comments
Assignees

Comments

@da-niao-dan
Copy link

The current send and recv is an grpc service. The grpc service limits the use of CPU to 1 core per connection. grpc/grpc-swift#992

image

This limit is by design, because grpc service considers the senario when a servie should handle multiple connections.

However, between two clusters, we may have only a handful connections that each needs to send a large object to the other end.
Sometimes we can superposition computation with such communications but sometimes it is hard to do so, particularly when communication time is much longer than computation time.

For example, we have two secretflow clusters, and one needs to send a large ciphertext to another. Using multiple CPUs to do send and receive can really speed up communications in this case.

Can you provide the feature to use multiple cores when doing send and receive?

@jovany-wang
Copy link
Collaborator

@NKcqx to triage.

@da-niao-dan
Copy link
Author

This feature is somewhat important in secretflow's HEU based algorithms. please help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants