feat(iroh-net): Add a Watchable struct for use in the Endpoint API #2806

flub · 2024-10-15T14:59:25Z

Description

This implements a Watchable struct and a Watcher which provides access to the watched value in several ways, including some streams.

Breaking Changes

iroh::Endpoint::conn_type_stream is renamed to iroh::Endpoint::conn_type and returns Result<Watcher<ConnectionType>> instead of Result<ConnectionTypeStream>
To migrate, use endpoint.conn_type()?.stream() instead of endpoint.conn_type_stream()?.
iroh::Endpoint::home_relay now returns Watcher<Option<RelayUrl>> instead of Option<RelayUrl>.
To migrate, use endpoint.home_relay().get()? instead of endpoint.home_relay().
removed iroh::Endpoint::watch_home_relay
To migrate, use endpoint.home_relay().initialized().await? instead of endpoint.watch_home_relay().next().await and
use endpoint.home_relay().stream() instead of endpoint.watch_home_relay().next().await.
removed DirectAddrsStream and ConnTypeStream.
Use iroh::watchable::WatcherStream for as named types instead.

Change checklist

Self-review.
Documentation updates following the style guide, if relevant.
Tests if relevant.
All breaking changes documented.

This adds a basic Watchable struct for use on the Endpoint API.

github-actions · 2024-10-15T15:02:00Z

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/2806/docs/iroh/

Last updated: 2024-12-13T09:40:10Z

mepi262 · 2024-11-08T14:05:56Z

@flub
Any update on this pull request?

flub · 2024-11-08T14:20:36Z

@mepi262 the current implementation is racy and needs fixing.

What makes you interested in this?

mepi262 · 2024-11-09T04:40:56Z

The reason in which I'm interested is followings.

I'm very much looking forward to the iroh v1.0.0.
This pull request is related to the issue, which is contained milestone v1.0.0
I'm worry about whether this pull request is merged, because this pull request has not been active since last month.
- People forget memory
- Merging pull request is difficult when time has gone, because of risk of conflict.

I wish you good health, good luck and success.

iroh-net/src/util/watchable.rs

matheus23 · 2024-11-20T11:42:02Z

Made some changes:

Added loom to watchable.rs (conditionally on the iroh_loom cfg)
Added a loom-based test that was failing in exactly the way @flub predicted
Fixed the race condition
Changed wakers from using a RwLock to using a Mutex, because we were only using RwLock::write
Removed Either by making initialized simpler

For anyone who's interested in running the loom test, you get some fun output like this:

LOOM_LOG=trace RUSTFLAGS="--cfg iroh_loom" cargo test --package iroh-net --lib -- util::watchable::tests::test_initialized_always_resolves --exact --nocapture

matheus23 · 2024-11-21T13:32:13Z

Should we (well, I) maybe address #2860 (comment) in here too?

What's the relationship between Endpoint::conn_type_stream and Connection::remote_address? It seems like remote_address should be deprecated and replaced with a method that returns a ConnectionType based on the current state of the connection.

flub

Looks good! Any reason this is still draft?

iroh-net/src/util/watchable.rs

flub · 2024-11-22T09:07:00Z

Should we (well, I) maybe address #2860 (comment) in here too?

What's the relationship between Endpoint::conn_type_stream and Connection::remote_address? It seems like remote_address should be deprecated and replaced with a method that returns a ConnectionType based on the current state of the connection.

I think this PR just introduces the watchable, follow-up PRs can start using it.

matheus23 · 2024-11-22T13:42:08Z

Looks good! Any reason this is still draft?

I think this PR just introduces the watchable, follow-up PRs can start using it.

Ah okay. Well my plan was to learn from the refactor of making iroh-net use this new watchable.

And I'm already finding things. E.g. do we want the Watchable to have shutdown functionality.

So, I'm still learning!
In any case, I'll split up the PR into watchable introduction & the refactor anyways.

flub · 2024-11-22T14:08:21Z

Looks good! Any reason this is still draft?

I think this PR just introduces the watchable, follow-up PRs can start using it.

Ah okay. Well my plan was to learn from the refactor of making iroh-net use this new watchable.

And I'm already finding things. E.g. do we want the Watchable to have shutdown functionality.

So, I'm still learning! In any case, I'll split up the PR into watchable introduction & the refactor anyways.

Makes sense as well. No big deal if you'd like to do this all in one place.

Cargo.toml

github-actions · 2024-12-10T10:55:54Z

Netsim report & logs for this PR have been generated and is available at: LOGS
This report will remain available for 3 days.

Last updated for commit: 5066045

…her` interface

…b(crate)` again

iroh/src/discovery.rs

iroh/src/discovery/pkarr.rs

iroh/src/watchable.rs

iroh/src/discovery/pkarr.rs

flub

LGTM

flub · 2024-12-13T10:07:45Z

iroh/src/endpoint.rs

    /// # Errors
    ///
    /// Will error if we do not have any address information for the given `node_id`.
-    pub fn conn_type_stream(&self, node_id: NodeId) -> Result<ConnectionTypeStream> {
-        self.msock.conn_type_stream(node_id)
+    pub fn conn_type(&self, node_id: NodeId) -> Result<Watcher<ConnectionType>> {


Probably out of scope for this PR. But what would folks think of changing fn conn_type(node_id) into fn network_path(node_id)? And Also rename ConnectionType into NetworkPath.

This name has always bothered me a bit, I'm not sure why. Maybe because this doesn't actually have much to do with a Connection, it is on a different level as shown by the fact this is an Endpoint API. But this is subtle and not very clear.

I like the idea :)
I have a bunch of follow-up PR ideas brewing anyways, maybe I'll just add them to the follow-up.

matheus23

Turning @flub's "LGTM" into a github-approved LGTM 🙃

matheus23 and others added 2 commits October 1, 2024 17:41

feat(iroh-net): Ported & modified Watcher implementation

70f61a8

feat(iroh-net): Add a Watchable struct

f946f07

This adds a basic Watchable struct for use on the Endpoint API.

flub requested a review from dignifiedquire October 15, 2024 15:01

flub requested a review from divagant-martian October 15, 2024 15:29

matheus23 mentioned this pull request Oct 25, 2024

Tracking: WebAssembly support for iroh-net #2799

Open

matheus23 reviewed Nov 20, 2024

View reviewed changes

iroh-net/src/util/watchable.rs Outdated Show resolved Hide resolved

matheus23 added 2 commits November 20, 2024 11:37

Merge branch 'main' into flub/watchable

9982640

Test with loom, fix race condition, switch to Mutex for wakers

daa3b67

Correct spelling

1141fc9

matheus23 added 2 commits November 21, 2024 14:34

Improved Watchable and Watcher impl & API

22498c2

Whoops fix test

be17d23

flub commented Nov 22, 2024

View reviewed changes

Compiles but fails tests

0e6bdf7

Merge branch 'main' into flub/watchable

a49e2a6

dignifiedquire reviewed Dec 6, 2024

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

Fix Watchable::set implementation

fe813ca

matheus23 self-assigned this Dec 10, 2024

matheus23 added this to the v0.30.0 milestone Dec 10, 2024

Address code review, make Watcher and Watchable available in pub API

4a24c89

Merge branch 'main' into flub/watchable

d287847

matheus23 added 6 commits December 11, 2024 16:16

Fix a possible race condition

675e04d

Adjust documentation to match implementation

a186583

Merge remote-tracking branch 'origin/main' into flub/watchable

4dea829

Convert DirectAddrStream into Watcher<Option<BTreeSet<DirectAddr>>>

f7d4c24

Merge Endpoint::{home_relay, watch_home_realy} together via a `Watc…

3f0f65b

…her` interface

Move iroh::util::watchable to iroh::watchable and make util `pu…

b76f11b

…b(crate)` again

matheus23 marked this pull request as ready for review December 11, 2024 17:29

matheus23 added 6 commits December 12, 2024 08:56

Adjust util::watchable imports to use watchable

90e825d

Merge remote-tracking branch 'origin/main' into flub/watchable

c6cde17

cargo +nightly clippy -p iroh --fix

c824573

cargo make format

8e485bb

Small fixes

1974629

Merge remote-tracking branch 'origin/main' into flub/watchable

890cb87

dignifiedquire reviewed Dec 12, 2024

View reviewed changes

iroh/src/discovery.rs Show resolved Hide resolved

dignifiedquire reviewed Dec 12, 2024

View reviewed changes

iroh/src/discovery/pkarr.rs Show resolved Hide resolved

matheus23 added 2 commits December 12, 2024 15:16

Add documentation about cancel safety and a small test

e26a7f9

Merge remote-tracking branch 'origin/main' into flub/watchable

bd58548

dignifiedquire reviewed Dec 12, 2024

View reviewed changes

iroh/src/watchable.rs Outdated Show resolved Hide resolved

matheus23 added 4 commits December 12, 2024 15:41

Better wording in documentation

6ca0f71

Remove unused imports

86bc24b

Fix doc links

37dead3

Fix newly introduced clippy lints

ac9d9d2

dignifiedquire reviewed Dec 12, 2024

View reviewed changes

iroh/src/discovery/pkarr.rs Outdated Show resolved Hide resolved

Use Watchable::default() when appropriate

6f078e9

flub commented Dec 13, 2024

View reviewed changes

matheus23 approved these changes Dec 13, 2024

View reviewed changes

matheus23 added this pull request to the merge queue Dec 13, 2024

Merged via the queue into main with commit 1a79a19 Dec 13, 2024
25 of 26 checks passed

dignifiedquire deleted the flub/watchable branch December 13, 2024 12:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(iroh-net): Add a Watchable struct for use in the Endpoint API #2806

feat(iroh-net): Add a Watchable struct for use in the Endpoint API #2806

flub commented Oct 15, 2024 •

edited by matheus23

Loading

github-actions bot commented Oct 15, 2024 •

edited

Loading

mepi262 commented Nov 8, 2024

flub commented Nov 8, 2024

mepi262 commented Nov 9, 2024

matheus23 commented Nov 20, 2024

matheus23 commented Nov 21, 2024 •

edited

Loading

flub left a comment

flub commented Nov 22, 2024

matheus23 commented Nov 22, 2024

flub commented Nov 22, 2024

github-actions bot commented Dec 10, 2024 •

edited

Loading

flub left a comment

flub Dec 13, 2024

matheus23 Dec 13, 2024

matheus23 left a comment

feat(iroh-net): Add a Watchable struct for use in the Endpoint API #2806

feat(iroh-net): Add a Watchable struct for use in the Endpoint API #2806

Conversation

flub commented Oct 15, 2024 • edited by matheus23 Loading

Description

Breaking Changes

Change checklist

github-actions bot commented Oct 15, 2024 • edited Loading

mepi262 commented Nov 8, 2024

flub commented Nov 8, 2024

mepi262 commented Nov 9, 2024

matheus23 commented Nov 20, 2024

matheus23 commented Nov 21, 2024 • edited Loading

flub left a comment

Choose a reason for hiding this comment

flub commented Nov 22, 2024

matheus23 commented Nov 22, 2024

flub commented Nov 22, 2024

github-actions bot commented Dec 10, 2024 • edited Loading

flub left a comment

Choose a reason for hiding this comment

flub Dec 13, 2024

Choose a reason for hiding this comment

matheus23 Dec 13, 2024

Choose a reason for hiding this comment

matheus23 left a comment

Choose a reason for hiding this comment

flub commented Oct 15, 2024 •

edited by matheus23

Loading

github-actions bot commented Oct 15, 2024 •

edited

Loading

matheus23 commented Nov 21, 2024 •

edited

Loading

github-actions bot commented Dec 10, 2024 •

edited

Loading