Skip to content
This repository has been archived by the owner on Aug 25, 2021. It is now read-only.

Implement dynamic tracks addition with renegotiation (#27) #105

Merged
merged 75 commits into from
Jun 26, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
e9a62ad
Add server-side renegotiation logic
evdokimovs May 21, 2020
9d63073
Add client side logic for renegotiation
evdokimovs May 21, 2020
5398103
Add code for testing renegotiation
evdokimovs May 21, 2020
311e592
Clean unused code
evdokimovs May 21, 2020
2e4d144
WIP
evdokimovs May 22, 2020
8e16b7c
Add debuging code
evdokimovs May 22, 2020
447f4bd
Fix renegotiation in the Firefox
evdokimovs May 25, 2020
b06a97a
Merge branch 'master' into renegotiation
evdokimovs May 25, 2020
3fc35e1
Add docs, refactor
evdokimovs May 25, 2020
42e7f5b
Merge branch 'master' into renegotiation
alexlapa May 29, 2020
510f038
refactor
alexlapa Jun 4, 2020
0f73f7a
WIP
evdokimovs Jun 4, 2020
c3c1736
Temporary revert broken commits
evdokimovs Jun 4, 2020
6420477
Remove PeerConnectionState and related things
evdokimovs Jun 4, 2020
9568c76
Add new renegotiation
evdokimovs Jun 4, 2020
de4ff6b
Implement tracks adding
evdokimovs Jun 4, 2020
019fafd
New impl for connect_endpoints
evdokimovs Jun 4, 2020
3074d28
Refactor
evdokimovs Jun 4, 2020
9df37c2
Refactor
evdokimovs Jun 5, 2020
52fd4c5
Take into account RenegotiationReason
evdokimovs Jun 5, 2020
29c6902
Pick room changes, fix Member interconnecting on Endpoint creating
evdokimovs Jun 5, 2020
d6c17d9
Cleanup
evdokimovs Jun 5, 2020
338abe5
connect_endpoints error handling
evdokimovs Jun 5, 2020
d0da210
Remove all unsafe code from actix_try_join_all
evdokimovs Jun 5, 2020
340fba6
Normal PeerSpec updating
evdokimovs Jun 5, 2020
f0bea8e
Refactor
evdokimovs Jun 5, 2020
b85372b
Fix tests
evdokimovs Jun 5, 2020
163847e
Add Event::TracksAdded to the RFC
evdokimovs Jun 8, 2020
afb7cb3
Add docs
evdokimovs Jun 8, 2020
3f8d8d6
Upd CHANGELOGs
evdokimovs Jun 8, 2020
d54f340
Reread [run ci]
evdokimovs Jun 8, 2020
eca043c
Temporary fix for the clippy [run ci]
evdokimovs Jun 8, 2020
c79aec5
Remove debug spec
evdokimovs Jun 8, 2020
499dad4
Fix linting [run ci]
evdokimovs Jun 9, 2020
21ea5d5
refactor
alexlapa Jun 9, 2020
258f371
Fix register_peer
evdokimovs Jun 10, 2020
a479312
Remove add_sink from the peers
evdokimovs Jun 10, 2020
7e8c1a3
Add E2E test
evdokimovs Jun 11, 2020
f383894
Add unit tests
evdokimovs Jun 11, 2020
31def5f
Refactor, fix broken Tracks adding
evdokimovs Jun 12, 2020
ac51f63
Add docs, refactor [run ci]
evdokimovs Jun 12, 2020
285d882
Final reread [run ci]
evdokimovs Jun 14, 2020
4997fd5
More stable E2E test [run ci]
evdokimovs Jun 15, 2020
85f630f
Add TracksApplied Event
evdokimovs Jun 17, 2020
4d00fb2
Refactor
evdokimovs Jun 17, 2020
104bbab
Fix traffic watcher Peer updating
evdokimovs Jun 17, 2020
b05090a
Test for the TrafficWatcher::register_peer in PeerService
evdokimovs Jun 17, 2020
76e43ab
Fix some docs, refactor
evdokimovs Jun 17, 2020
e118bd8
Fix tests
evdokimovs Jun 17, 2020
862d4bc
Improve docs, fix CHANGELOGs and RFC
evdokimovs Jun 17, 2020
18dc402
Minor doc [run ci]
evdokimovs Jun 17, 2020
21f5030
Remove unwrap [run ci]
evdokimovs Jun 17, 2020
fc41bdd
Meh, fix mistype [run ci]
evdokimovs Jun 17, 2020
0265e56
Merge branch 'master' into renegotiation
alexlapa Jun 17, 2020
1613971
refactor, add todos
alexlapa Jun 18, 2020
d2101b6
Fix renegotiation and Peer creation
evdokimovs Jun 18, 2020
eaa37d6
Refactor
evdokimovs Jun 18, 2020
e7ceafe
Get rid of the ActFuture in the PeersService
evdokimovs Jun 19, 2020
0a32df9
Cleanup and PeerMetricsService fix
evdokimovs Jun 19, 2020
5acdca0
Fix PeersService unit tests
evdokimovs Jun 19, 2020
3eda1aa
Fix E2E tests and signalling bug
evdokimovs Jun 19, 2020
8aacec6
Wrap PeersService to Rc
evdokimovs Jun 19, 2020
e5076ca
Use async fns in the PeersService
evdokimovs Jun 19, 2020
f5b5d1a
Fix lints, refactor
evdokimovs Jun 22, 2020
4f08797
Merge branch 'master' into renegotiation
evdokimovs Jun 22, 2020
721f08e
Update docs
evdokimovs Jun 22, 2020
c6661af
Reread [run ci]
evdokimovs Jun 22, 2020
2b0ee46
Fix import [run ci]
evdokimovs Jun 22, 2020
f7f5917
Merge branch 'master' into renegotiation [run ci]
evdokimovs Jun 22, 2020
2b31986
Merge branch 'master' into renegotiation [run ci]
evdokimovs Jun 23, 2020
f1b49d0
Minor fixes [run ci]
evdokimovs Jun 23, 2020
3cf520c
refactor [run ci]
alexlapa Jun 25, 2020
3368a0e
Spawn send_peer_created on ctx [run ci]
evdokimovs Jun 25, 2020
87fadf5
refactor [run ci]
alexlapa Jun 25, 2020
854744e
Corrections
tyranron Jun 26, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ All user visible changes to this project will be documented in this file. This p
- Send reason of closing WebSocket connection as [Close](https://tools.ietf.org/html/rfc4566#section-5.14) frame's description ([#58]);
- Send `Event::RpcSettingsUpdated` when `Member` connects ([#75]);
- Send relay mode in `Event::PeerCreated` which is used for configuring client's `RtcIceTransportPolicy` ([#79]);
- Send `Command::UpdateTracks` on `Event::TracksUpdated` ([#81]).
- Emit `TracksApplied` event to create new and update existing tracks ([#105]);
- `PeerConnection` renegotiation functionality ([#105]).
- [Coturn] integration:
- [Coturn] sessions destroying ([#84]);
- [Coturn] stats processing ([#94]).
Expand All @@ -55,7 +56,8 @@ All user visible changes to this project will be documented in this file. This p
### Fixed

- Signalling:
- Room crashing when handling commands with non-existent `peer_id` ([#86]).
- Room crashing when handling commands with non-existent `peer_id` ([#86]);
- Adding new endpoints to the already interconnected `Member`s ([#105]).

[#28]: /../../pull/28
[#33]: /../../pull/33
Expand All @@ -70,6 +72,7 @@ All user visible changes to this project will be documented in this file. This p
[#94]: /../../pull/94
[#95]: /../../pull/95
[#98]: /../../pull/98
[#105]: /../../pull/105



Expand Down
8 changes: 4 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -703,19 +703,6 @@ It's recommended to cache `Peer` ID and `Member` ID relations in `Web Client`'s
```
</details>

#### 10. TracksUpdated

`Media Server` notifies about necessity to update `Track`s in specified `Peer`.

Can be used to update existing `Track` settings (e.g. change to lower video resolution, mute audio).

```rust
struct TracksUpdated {
peer_id: PeerId,
tracks_patches: Vec<TrackPatch>,
}
```


### Commands

Expand Down Expand Up @@ -1102,7 +1089,7 @@ Metrics list will be extended as needed.

#### 10. UpdateTracks

`Web Client` asks permission to update `Track`s in specified `Peer`. `Media Server` gives permission by sending `Event::TracksUpdated`.
`Web Client` asks permission to update `Track`s in specified `Peer`. `Media Server` gives permission by sending `Event::TracksApplied`.

```rust
struct UpdateTracks {
Expand Down
5 changes: 4 additions & 1 deletion jason/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@ All user visible changes to this project will be documented in this file. This p
- Emitting of RPC commands:
- `AddPeerConnectionMetrics` with `IceConnectionState` and `PeerConnectionState` ([#71], [#87]);
- `ApplyTracks` for muting/unmuting ([#81]);
- `AddPeerConnectionStats` with `RtcStats` ([#90]).
- `AddPeerConnectionStats` with `RtcStats` ([#90]);
- Handling of RPC events:
- `TracksApplied` ([#105]).
- Error handling:
- Library API:
- `JasonError` as library error with trace information and underlying JS error if it is the cause ([#55])
Expand All @@ -80,6 +82,7 @@ All user visible changes to this project will be documented in this file. This p
[#87]: /../../pull/87
[#90]: /../../pull/90
[#97]: /../../pull/97
[#105]: /../../pull/105
[#106]: /../../pull/106


Expand Down
118 changes: 106 additions & 12 deletions jason/src/api/room.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ use futures::{
use js_sys::Promise;
use medea_client_api_proto::{
Command, Direction, Event as RpcEvent, EventHandler, IceCandidate,
IceConnectionState, IceServer, PeerConnectionState, PeerId, PeerMetrics,
Track, TrackId, TrackPatch,
IceConnectionState, IceServer, NegotiationRole, PeerConnectionState,
PeerId, PeerMetrics, Track, TrackId, TrackPatch, TrackUpdate,
};
use tracerr::Traced;
use wasm_bindgen::{prelude::*, JsValue};
Expand Down Expand Up @@ -780,7 +780,7 @@ impl EventHandler for InnerRoom {
fn on_peer_created(
&mut self,
peer_id: PeerId,
sdp_offer: Option<String>,
negotiation_role: NegotiationRole,
tracks: Vec<Track>,
ice_servers: Vec<IceServer>,
is_force_relayed: bool,
Expand All @@ -804,13 +804,16 @@ impl EventHandler for InnerRoom {

self.create_connections_from_tracks(&tracks);

// TODO(alexlapa): Eliminate code duplication (on_tracks_applied).
// Doing Room refactoring in another PR, it`ll allow
// to fix this smoothly.
let local_stream_constraints = self.local_stream_settings.clone();
let rpc = Rc::clone(&self.rpc);
let error_callback = Rc::clone(&self.on_failed_local_stream);
spawn_local(
async move {
match sdp_offer {
None => {
match negotiation_role {
NegotiationRole::Offerer => {
let sdp_offer = peer
.get_offer(tracks, local_stream_constraints)
.await
Expand All @@ -826,7 +829,7 @@ impl EventHandler for InnerRoom {
mids,
});
}
Some(offer) => {
NegotiationRole::Answerer(offer) => {
let sdp_answer = peer
.process_offer(
offer,
Expand Down Expand Up @@ -916,16 +919,107 @@ impl EventHandler for InnerRoom {
});
}

/// Updates [`Track`]s of this [`Room`].
fn on_tracks_updated(&mut self, peer_id: PeerId, tracks: Vec<TrackPatch>) {
if let Some(peer) = self.peers.get(peer_id) {
if let Err(err) = peer.update_senders(tracks) {
JasonError::from(err).print();
}
/// Creates new `Track`s, updates existing [`Sender`]s/[`Receiver`]s with
/// [`TrackUpdate`]s.
///
/// Will start renegotiation process if `Some` [`NegotiationRole`] is
/// provided.
fn on_tracks_applied(
&mut self,
peer_id: PeerId,
updates: Vec<TrackUpdate>,
negotiation_role: Option<NegotiationRole>,
) {
let peer = if let Some(peer) = self.peers.get(peer_id) {
peer
} else {
JasonError::from(tracerr::new!(RoomError::NoSuchPeer(peer_id)))
.print();
return;
};

let mut new_tracks = Vec::new();
let mut patches = Vec::new();

for update in updates {
match update {
TrackUpdate::Added(track) => {
new_tracks.push(track);
}
TrackUpdate::Updated(track_patch) => {
patches.push(track_patch);
}
}
}
if let Err(err) = peer.update_senders(patches) {
JasonError::from(err).print();
return;
}

// TODO(alexlapa): Eliminate code duplication (on_peer_created).
// Doing Room refactoring in another PR, it`ll allow
// to fix this smoothly.
let local_stream_constraints = self.local_stream_settings.clone();
let rpc = Rc::clone(&self.rpc);
let error_callback = Rc::clone(&self.on_failed_local_stream);
spawn_local(
async move {
match negotiation_role {
None => {
peer.create_tracks(new_tracks)
.await
.map_err(tracerr::map_from_and_wrap!())?;
}
Some(NegotiationRole::Offerer) => {
let sdp_offer = peer
.get_offer(new_tracks, local_stream_constraints)
.await
.map_err(tracerr::map_from_and_wrap!())?;
let mids = peer
.get_mids()
.map_err(tracerr::map_from_and_wrap!())?;
let senders_statuses = peer.get_senders_statuses();
rpc.send_command(Command::MakeSdpOffer {
peer_id,
sdp_offer,
senders_statuses,
mids,
});
}
Some(NegotiationRole::Answerer(offer)) => {
let sdp_answer = peer
.process_offer(
offer,
new_tracks,
local_stream_constraints,
)
.await
.map_err(tracerr::map_from_and_wrap!())?;
let senders_statuses = peer.get_senders_statuses();
rpc.send_command(Command::MakeSdpAnswer {
peer_id,
sdp_answer,
senders_statuses,
});
}
};
Result::<_, Traced<RoomError>>::Ok(())
}
.then(|result| async move {
if let Err(err) = result {
let (err, trace) = err.into_parts();
match err {
RoomError::InvalidLocalStream(_)
| RoomError::CouldNotGetLocalMedia(_) => {
let e = JasonError::from((err, trace));
e.print();
error_callback.call(e);
}
_ => JasonError::from((err, trace)).print(),
};
};
}),
)
}
}

Expand Down
8 changes: 2 additions & 6 deletions jason/src/peer/media/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -243,18 +243,14 @@ impl MediaConnections {
out
}

/// Synchronizes local state with provided tracks. Creates new [`Sender`]s
/// and [`Receiver`]s for each new [`Track`], and updates [`Track`] if
/// its settings has been changed.
/// Creates new [`Sender`]s and [`Receiver`]s for each new [`Track`].
///
/// # Errors
///
/// With [`MediaConnectionsError::TransceiverNotFound`] if could not create
/// new [`Sender`] cause transceiver with specified `mid` does not
/// exist.
// TODO: Doesnt really updates anything, but only generates new senders
// and receivers atm.
pub fn update_tracks<I: IntoIterator<Item = Track>>(
pub fn create_tracks<I: IntoIterator<Item = Track>>(
&self,
tracks: I,
) -> Result<()> {
Expand Down
25 changes: 20 additions & 5 deletions jason/src/peer/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -576,7 +576,7 @@ impl PeerConnection {
local_stream: Option<MediaStreamSettings>,
) -> Result<String> {
self.media_connections
.update_tracks(tracks)
.create_tracks(tracks)
.map_err(tracerr::map_from_and_wrap!())?;

self.update_local_stream(local_stream)
Expand All @@ -588,10 +588,22 @@ impl PeerConnection {
.create_and_set_offer()
.await
.map_err(tracerr::map_from_and_wrap!())?;

Ok(offer)
}

/// Creates new [`Sender`]s and [`Receiver`]s for each new [`Track`].
///
/// # Errors
///
/// With [`MediaConnectionsError::TransceiverNotFound`] if could not create
/// new [`Sender`] because transceiver with specified `mid` doesn't exist.
pub async fn create_tracks(&self, tracks: Vec<Track>) -> Result<()> {
self.media_connections
.create_tracks(tracks)
.map_err(tracerr::map_from_and_wrap!())?;
Ok(())
}

/// Inserts provided [MediaStream][1] into underlying [RTCPeerConnection][2]
/// if it has all required tracks.
/// Requests local stream from [`MediaManager`] if no stream was provided.
Expand Down Expand Up @@ -775,17 +787,20 @@ impl PeerConnection {
Direction::Recv { .. } => true,
});

// update receivers
// create receivers
self.media_connections
.update_tracks(recv)
.create_tracks(recv)
.map_err(tracerr::map_from_and_wrap!())?;

// set offer, which will create transceivers and discover remote tracks
// in receivers
self.set_remote_offer(offer)
.await
.map_err(tracerr::wrap!())?;

// create senders
self.media_connections
.update_tracks(send)
.create_tracks(send)
.map_err(tracerr::map_from_and_wrap!())?;

self.update_local_stream(local_constraints)
Expand Down
Loading