Skip to content

Commit

Permalink
⚡️ WebGear_RTC: Implemented a new easy way of defining Custom Streami…
Browse files Browse the repository at this point in the history 
…ng Class [#272]

- 💥 Removed support for assigning Custom Media Server Class(inherited from aiortc's VideoStreamTrack) in WebGear_RTC through its `config` global parameter.
- ✨ Added new `custom_stream` attribute with WebGear_RTC `options` parameter that allows you to easily define your own Custom Streaming Class with suitable source(such as OpenCV).
- ⚡️ This implementation supports repeated Auto-Reconnection or Auto-Refresh out-of-the-box.
- 🧑‍💻 This implementation is more user-friendly and easy to integrate within complex APIs.
- 🎨 This implementation supports all vidgear's VideoCapture APIs readily as input.
- 🏗️ This implementation requires at-least `read()` and `stop()` methods implemented within Custom Streaming Class, otherwise WebGear_RTC will throw ValueError.
- 🥚 WebGear_RTC API now automatically constructs `av.frame.Frame` from `numpy.nd.array` based on available channels in frames.
- 🏗️ WebGear_RTC API will now throws ValueError if `source` parameter is None and `custom_stream` attribute isn't defined.

- CI:
  - 👷 Updated CI tests for new WebGear_RTC custom streaming class.
  - 👷 Disabled test_stream_mode test until issue #273 is resolved.
  - 🐛 Fixed NameError bugs in WebGear_RTC CI test.

- Docs:
  - 📝 Added related usage docs for new WebGear_RTC custom streaming class.
  - 📝 Updated Advanced examples using WebGear_RTC's custom streaming class.
  - 👽️ Added changes for upgrading mkdocs-material from 7.x to 8.x in docs.
  - 🚸 Added outdated version warning block.
  - 🐛 Fixed content tabs failing to work.
  - 🚩 Updated WebGear_RTC parameters.
  - 🔥 Removed slugify from mkdocs causing invalid hyperlinks in docs.
  - 🐛 Fixed hyperlink in announcement bar.
  - 💄 Updated code highlighting.
  • Loading branch information
@abhiTronix
abhiTronix authored Dec 2, 2021
2 parents 0cb52d1 + c0c66e5 commit cd481c1
Show file tree
Hide file tree
Showing 10 changed files with 434 additions and 382 deletions.
105 changes: 52 additions & 53 deletions docs/gears/webgear_rtc/advanced.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,87 +64,85 @@ web.shutdown()

## Using WebGear_RTC with a Custom Source(OpenCV)

WebGear_RTC allows you to easily define your own Custom Media Server with a custom source that you want to use to transform your frames before sending them onto the browser.
WebGear_RTC provides [`custom_stream`](../params/#webgear_rtc-specific-attributes) attribute with its `options` parameter that allows you to easily define your own Custom Streaming Class with suitable source that you want to use to transform your frames before sending them onto the browser.

Let's implement a bare-minimum example with a Custom Source using WebGear_RTC API and OpenCV:

??? new "New in v0.2.4"
This implementation was added in `v0.2.4`.

!!! warning "Auto-Reconnection will NOT work with Custom Source(OpenCV)"
!!! success "Auto-Reconnection or Auto-Refresh works out-of-the-box with this implementation."

- The WebGear_RTC's inbuilt auto-reconnection feature only available with its internal [**RTC_VideoServer**](https://github.com/abhiTronix/vidgear/blob/38a7f54eb911218e1fd6a95e243da2fba51a6991/vidgear/gears/asyncio/webgear_rtc.py#L77) which can seamlessly handle restarting of source/server any number of times, and therefore will fail to work with your user-defined Custom Source(OpenCV).
- This means that once the browser tab with WebGear_RTC stream is closed, it will require you to manually close and restart the WebGear_RTC server, in order to refresh or reopen it in a new browser/tab successfully.
!!! danger "Make sure your Custom Streaming Class at-least implements `read()` and `stop()` methods as shown in following example, otherwise WebGear_RTC will throw ValueError!"

!!! danger "Make sure your Custom Media Server Class is inherited from aiortc's [VideoStreamTrack](https://github.com/aiortc/aiortc/blob/a270cd887fba4ce9ccb680d267d7d0a897de3d75/src/aiortc/mediastreams.py#L109) only and at-least implements `recv()` and `terminate()` methods as shown in following example, otherwise WebGear_RTC will throw ValueError!"
???+ tip "Using Vidgear's VideoCapture APIs instead of OpenCV"
You can directly replace Custom Streaming Class(`Custom_Stream_Class` in following example) with any [VideoCapture APIs](../../#a-videocapture-gears). These APIs implements `read()` and `stop()` methods by-default, so they're also supported out-of-the-box.

See this [example ➶](../../../help/screengear_ex/#using-screengear-with-webgear_rtc) for more information.

```python hl_lines="13-63 67"

```python hl_lines="6-54 58"
# import necessary libs
import uvicorn, asyncio, cv2
from av import VideoFrame
from aiortc import VideoStreamTrack
from aiortc.mediastreams import MediaStreamError
import uvicorn, cv2
from vidgear.gears.asyncio import WebGear_RTC
from vidgear.gears.asyncio.helper import reducer

# initialize WebGear_RTC app without any source
web = WebGear_RTC(logging=True)

# create your own Bare-Minimum Custom Media Server
class Custom_RTCServer(VideoStreamTrack):
# create your own custom streaming class
class Custom_Stream_Class:
"""
Custom Media Server using OpenCV, an inherit-class
to aiortc's VideoStreamTrack.
Custom Streaming using OpenCV
"""

def __init__(self, source=None):
def __init__(self, source=0):

# don't forget this line!
super().__init__()
# !!! define your own video source here !!!
self.source = cv2.VideoCapture(source)

# initialize global params
self.stream = cv2.VideoCapture(source)
# define running flag
self.running = True

async def recv(self):
"""
A coroutine function that yields `av.frame.Frame`.
"""
# don't forget this function!!!
def read(self):

# get next timestamp
pts, time_base = await self.next_timestamp()
# don't forget this function!!!

# read video frame
(grabbed, frame) = self.stream.read()
# check if source was initialized or not
if self.source is None:
return None
# check if we're still running
if self.running:
# read frame from provided source
(grabbed, frame) = self.source.read()
# check if frame is available
if grabbed:

# if NoneType
if not grabbed:
return MediaStreamError
# do something with your OpenCV frame here

# reducer frames size if you want more performance otherwise comment this line
frame = await reducer(frame, percentage=30) # reduce frame by 30%
# lets convert frame to gray for this example
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

# contruct `av.frame.Frame` from `numpy.nd.array`
av_frame = VideoFrame.from_ndarray(frame, format="bgr24")
av_frame.pts = pts
av_frame.time_base = time_base
# return our gray frame
return gray
else:
# signal we're not running now
self.running = False
# return None-type
return None

# return `av.frame.Frame`
return av_frame
def stop(self):

def terminate(self):
"""
Gracefully terminates VideoGear stream
"""
# don't forget this function!!!

# terminate
if not (self.stream is None):
self.stream.release()
self.stream = None
# flag that we're not running
self.running = False
# close stream
if not self.source is None:
self.source.release()

# assign your Custom Streaming Class with adequate source (for e.g. foo.mp4)
# to `custom_stream` attribute in options parameter
options = {"custom_stream": Custom_Stream_Class(source="foo.mp4")}

# assign your custom media server to config with adequate source (for e.g. foo.mp4)
web.config["server"] = Custom_RTCServer(source="foo.mp4")
# initialize WebGear_RTC app without any source
web = WebGear_RTC(logging=True, **options)

# run this app on Uvicorn server at address http://localhost:8000/
uvicorn.run(web(), host="localhost", port=8000)
Expand All @@ -155,6 +153,7 @@ web.shutdown()

**And that's all, Now you can see output at [`http://localhost:8000/`](http://localhost:8000/) address.**


 


Expand Down
22 changes: 22 additions & 0 deletions docs/gears/webgear_rtc/params.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,28 @@ This parameter can be used to pass user-defined parameter to WebGear_RTC API by

### WebGear_RTC Specific attributes

* **`custom_stream`** _(class)_ : Can be used to easily define your own Custom Streaming Class with suitable custom source(such as OpenCV) that you want to use to transform your frames before sending them onto the browser.

!!! danger "Make sure your Custom Streaming Class at-least implements `read()` and `stop()` methods, otherwise WebGear_RTC will throw ValueError!"

??? new "New in v0.2.4"
This attribute was added in `v0.2.4`.

??? tip "Using Vidgear's VideoCapture APIs instead of OpenCV"
You can directly replace Custom Streaming Class with any [VideoCapture APIs](../../#a-videocapture-gears). These APIs implements `read()` and `stop()` methods by-default, so they're also supported out-of-the-box.

See this [example ➶](../../../help/screengear_ex/#using-screengear-with-webgear_rtc) for more information.

!!! example "Its complete usage example is given [here ➶](../advanced/#using-webgear_rtc-with-a-custom-sourceopencv)."

```python
# set CamGear as custom streaming class with adequate parameters
options = {"custom_stream": CamGear(source="foo.mp4", logging=True)}
# assign it
WebGear_RTC(logging=True, **options)
```


* **`custom_data_location`** _(string)_ : Can be used to change/alter [*default location*](../overview/#default-location) path to somewhere else. Its usage is as follows:

```python
Expand Down
135 changes: 70 additions & 65 deletions docs/help/netgear_ex.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -178,8 +178,8 @@ server.close()

The complete usage example is as follows:

??? new "New in v0.2.2"
This example was added in `v0.2.2`.
??? new "New in v0.2.4"
This example was added in `v0.2.4`.

### Client + WebGear_RTC Server

Expand All @@ -193,33 +193,19 @@ Open a terminal on Client System where you want to display the input frames _(an

!!! info "Note down the local IP-address of this system(required at Server's end) and also replace it in the following code. You can follow [this FAQ](../netgear_faqs/#how-to-find-local-ip-address-on-different-os-platforms) for this purpose."

```python
# import required libraries
import uvicorn, asyncio, cv2
from av import VideoFrame
from aiortc import VideoStreamTrack
from aiortc.mediastreams import MediaStreamError
!!! fail "For VideoCapture APIs you also need to implement `start()` in addition to `read()` and `stop()` methods in your Custom Streaming Class as shown in following example, otherwise WebGear_RTC will fail to work!"

```python hl_lines="8-79 92-101"
# import necessary libs
import uvicorn, cv2
from vidgear.gears import NetGear
from vidgear.gears.helper import reducer
from vidgear.gears.asyncio import WebGear_RTC
from vidgear.gears.asyncio.helper import reducer

# initialize WebGear_RTC app without any source
web = WebGear_RTC(logging=True)

# activate jpeg encoding and specify other related parameters
options = {
"jpeg_compression": True,
"jpeg_compression_quality": 90,
"jpeg_compression_fastdct": True,
"jpeg_compression_fastupsample": True,
}


# create your own Bare-Minimum Custom Media Server
class Custom_RTCServer(VideoStreamTrack):
# create your own custom streaming class
class Custom_Stream_Class:
"""
Custom Media Server using OpenCV, an inherit-class
to aiortc's VideoStreamTrack.
Custom Streaming using NetGear Receiver
"""

def __init__(
Expand All @@ -229,72 +215,91 @@ class Custom_RTCServer(VideoStreamTrack):
protocol="tcp",
pattern=1,
logging=True,
options={},
**options,
):
# don't forget this line!
super().__init__()

# initialize global params
# Define NetGear Client at given IP address and define parameters
self.client = NetGear(
receive_mode=True,
address=address,
port=protocol,
port=port,
protocol=protocol,
pattern=pattern,
receive_mode=True,
logging=logging,
**options
)
self.running = False

async def recv(self):
"""
A coroutine function that yields `av.frame.Frame`.
"""
def start(self):

# don't forget this function!!!
# This function is specific to VideoCapture APIs only

# get next timestamp
pts, time_base = await self.next_timestamp()
if not self.source is None:
self.source.start()

# receive frames from network
frame = self.client.recv()
def read(self):

# if NoneType
if frame is None:
raise MediaStreamError
# don't forget this function!!!

# reducer frames size if you want more performance otherwise comment this line
frame = await reducer(frame, percentage=30) # reduce frame by 30%
# check if source was initialized or not
if self.source is None:
return None
# check if we're still running
if self.running:
# receive frames from network
frame = self.client.recv()
# check if frame is available
if not (frame is None):

# do something with your OpenCV frame here

# contruct `av.frame.Frame` from `numpy.nd.array`
av_frame = VideoFrame.from_ndarray(frame, format="bgr24")
av_frame.pts = pts
av_frame.time_base = time_base
# reducer frames size if you want more performance otherwise comment this line
frame = reducer(frame, percentage=20) # reduce frame by 20%

# return `av.frame.Frame`
return av_frame
# return our gray frame
return frame
else:
# signal we're not running now
self.running = False
# return None-type
return None

def stop(self):

def terminate(self):
"""
Gracefully terminates VideoGear stream
"""
# don't forget this function!!!

# terminate
# flag that we're not running
self.running = False
# close stream
if not (self.client is None):
self.client.close()
self.client = None


# assign your custom media server to config with adequate IP address
# !!! change following IP address '192.168.x.xxx' with yours !!!
web.config["server"] = Custom_RTCServer(
address="192.168.x.xxx",
port="5454",
protocol="tcp",
pattern=1,
logging=True,
**options
)
# activate jpeg encoding and specify NetGear related parameters
options = {
"jpeg_compression": True,
"jpeg_compression_quality": 90,
"jpeg_compression_fastdct": True,
"jpeg_compression_fastupsample": True,
}

# assign your Custom Streaming Class with adequate NetGear parameters
# to `custom_stream` attribute in options parameter of WebGear_RTC.
options = {
"custom_stream": Custom_Stream_Class(
address="192.168.x.xxx",
port="5454",
protocol="tcp",
pattern=1,
logging=True,
**options
)
}

# initialize WebGear_RTC app without any source
web = WebGear_RTC(logging=True, **options)

# run this app on Uvicorn server at address http://localhost:8000/
uvicorn.run(web(), host="localhost", port=8000)
Expand Down
Loading

0 comments on commit cd481c1

Please sign in to comment.