Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(iroh-net): Removeable discovery #2827

Closed
wants to merge 2 commits into from

Conversation

rklaehn
Copy link
Contributor

@rklaehn rklaehn commented Oct 23, 2024

Description

Adds a version of ConcurrentDiscovery where the individual discovery services are identified by ids.

That way you can dynamically add them, remove them, and even get a reference.

Breaking Changes

None, this just adds something. In the long term we would probably deprecate and then remove ConcurrentDiscovery however, since the functionality is quite overlapping.

Notes & open questions

The current implementation uses an Arc<Mutex<HashMap>>. When using the services in the service trait implementation, we hold the lock while e.g. calling publish. This is OK for a properly implemented discovery service that does not do anything expensive during publish, but you could come up with a convoluted discovery impl that would e.g. try to remove a discovery service during publish, which would cause a deadlock. Also, people might block in publish, meaning that another code path that calls remove would be blocked.

Sooo. The safe solution for this is to not hold the lock at all while executing code that can be influenced by the user. To do that there are two approaches:

  • make a deep copy. Not the end of the world, but also not nice. You would allocate during every call to resolve.
  • use a persistent data structure for the inside of the Arc<Mutex<...>> and do a zero cost snapshot. This means having a dependency on rpds.

What do you think?

  1. hope that people won't do totally broken and convoluted impls of discovery
  2. do a defensive deep copy and eat the overhead
  3. use persistent data structures

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.
  • Tests if relevant.
  • All breaking changes documented.

this creates a map of discovery services that can be used to get the discovery
services and/or dynamically modify them. The downside is that it now requires
a RwLock, and there is a potential for a deadlock if people write a very
convoluted discovery service...
Copy link

github-actions bot commented Oct 23, 2024

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/2827/docs/iroh/

Last updated: 2024-11-20T10:48:44Z

Copy link

github-actions bot commented Oct 23, 2024

Netsim report & logs for this PR have been generated and is available at: LOGS
This report will remain available for 3 days.

Last updated for commit: fc640ca

@dignifiedquire
Copy link
Contributor

if you just store an Arc<DiscoveryServiceEntry> you can clone super cheap, without a lot of overhead

@rklaehn rklaehn changed the base branch from main to static-discovery October 23, 2024 10:28
@rklaehn
Copy link
Contributor Author

rklaehn commented Oct 23, 2024

if you just store an Arc<DiscoveryServiceEntry> you can clone super cheap, without a lot of overhead

Yeah, but then I still have to make a copy of the Vec that contains them. I already had a version where I was using a smallvec for this, but using a persistent data structure seems like a better solution.

@rklaehn rklaehn changed the title Removeable discovery feat(iroh-net): Removeable discovery Oct 24, 2024
Copy link
Contributor

@flub flub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rather like this, maybe open the PR and ask @ramfox for input as well?

What I do wonder is how feasible it is to add this functionality to the existing ConcurrentDiscovery instead of creating a new struct that will eventually replace it?

.field("max_id", &self.max_id)
.finish_non_exhaustive()
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not derrive_more?

iroh-net/src/discovery.rs Outdated Show resolved Hide resolved
Co-authored-by: Floris Bruynooghe <flub@n0.computer>
@rklaehn rklaehn deleted the branch static-discovery November 20, 2024 12:31
@rklaehn rklaehn closed this Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants