Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Feat/memory transport #3022

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

Conversation

pyropy
Copy link

@pyropy pyropy commented Oct 24, 2024

Closes #2293

@MarcoPolo
Copy link
Collaborator

Friendly reminder to test your transport by adding a case here:

var transportsToTest = []TransportTestCase{

@pyropy
Copy link
Author

pyropy commented Oct 24, 2024

Friendly reminder to test your transport by adding a case here:

var transportsToTest = []TransportTestCase{

Thanks, will add it.

Could you possibly take a look at this PR multiformats/go-multiaddr#256 ?

Edit: This is the issue I'm currently having while running my tests

Error:       Received unexpected error:
        	   failed to parse multiaddr "/memory/1234": unknown protocol memory

@pyropy
Copy link
Author

pyropy commented Nov 6, 2024

I have few questions:

  • When OpenStream is called on the Connection, how is the returned stream used? Is the AcceptStream called after OpenStream? My concern is about how the channels would be exchanged between the opened streams for the connection. I guess if OpenStream call is followed by AcceptStream we can send stream to "remote" connection.
  • Should we keep using io.Pipe or switch it up with channels? I've seen that io.Pipe is using channels in the background.

@sukunrt
Copy link
Member

sukunrt commented Nov 11, 2024

When OpenStream is called on the Connection, how is the returned stream used? Is the AcceptStream called after OpenStream? My concern is about how the channels would be exchanged between the opened streams for the connection. I guess if OpenStream call is followed by AcceptStream we can send stream to "remote" connection.

OpenStream and AcceptStream happen on different libp2p hosts. Consider two hosts A and B. When A calls OpenStream, it creates a stream it can write to and read from. B will get this stream when it calls AcceptStream on the connection.
In the swarm, AcceptStream happens automatically because every open connection is waiting on AcceptStream here: https://github.com/libp2p/go-libp2p/blob/master/p2p/net/swarm/swarm_conn.go#L118
For implementing a transport, the transport should buffer streams till they are accepted by calling AcceptStream.

Should we keep using io.Pipe or switch it up with channels? I've seen that io.Pipe is using channels in the background.

I think io.Pipe is fine. This transport will only be used for tests, so it's nice to get as much of the work done by the stdlib as we can.

@pyropy pyropy force-pushed the feat/memory-transport branch from b14b377 to e3203f2 Compare November 21, 2024 10:24
@pyropy pyropy force-pushed the feat/memory-transport branch from 76e141f to f511c2d Compare November 25, 2024 10:58
@pyropy pyropy marked this pull request as ready for review November 28, 2024 17:33
@pyropy
Copy link
Author

pyropy commented Nov 28, 2024

I'm trying to get tests to pass they keep failing. For example, TestPing/Memory fails at protocol negotiation with either failed to negotiate protocol: io: read/write on closed pipe or failed to negotiate protocol: EOF. My guess is that it's failing at the stream level.

- Return io.EOF when reading if remote stream is closed
- Return network.ErrReset when reading/writing if remote stream is reset
- Return io.ErrPipeClosed when reading if remote stream is closed
Copy link
Member

@sukunrt sukunrt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the tests are specifically failing because you don't support deadlines and basic_host expects deadline support. https://github.com/libp2p/go-libp2p/blob/master/p2p/host/basic/basic_host.go#L448

@@ -0,0 +1 @@
package memory
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Write these before transport integration tests.

t.Errorf("expected to be able to dial memory address (%s)", validAddr)
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Write end to end transport tests here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

transport: add in memory transport
3 participants