Stream resize error

[ioangatop]

Hi! There is an issue with frame fetching from a youtube video when using different resolutions.

Code to reproduce:

from vidgear.gears.helper import dimensions_to_resolutions
from vidgear.gears import VideoGear

import pafy
import cv2

source = "https://youtu.be/j1GLs_fMn1s"

video = pafy.new(source) 
valid_streams = dimensions_to_resolutions(
    [stream.resolution for stream in video.allstreams]
)   # ['144p', '240p', '360p', '480p', '720p', '1080p']

options = {
    "STREAM_RESOLUTION": valid_streams[1],  # '240p'
    "STREAM_PARAMS": {"nocheckcertificate": True}
}

stream = VideoGear(
    source=source,
    stream_mode=True,
    **options
).start()

while True:
    frame = stream.read()

    if frame is None:
        break

    cv2.imwrite('frame.jpg', frame)

    break

stream.stop()

Fetched image w/ 240p resolution (happens with 480p as well):

Fetched image w/ 144p resolution:

Logs

10:17:21 :: CamGear :: INFO :: Using Youtube-dl Backend
[ WARN:0] global /opt/opencv-4.5.1/modules/videoio/src/cap_gstreamer.cpp (924) open OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:0] global /opt/opencv-4.5.1/modules/videoio/src/cap_gstreamer.cpp (961) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=449, duration=-1
10:17:22 :: CamGear :: DEBUG :: Setting backend `1800` for this source.

[welcome]

Thanks for opening this issue, a maintainer will get back to you shortly!

In the meantime:

• Read our Issue Guidelines, and update your issue accordingly. Please note that your issue will be fixed much faster if you spend about half an hour preparing it, including the exact reproduction steps and a demo.
• Go comprehensively through our dedicated FAQ & Troubleshooting section.
• For any quick questions and typos, please refrain from opening an issue, as you can reach us on Gitter community channel.

[abhiTronix]

@ioangatop Yes bug is reproducible. But bug is with OpenCV's GSTREAMER backend and not vidgear. You can test this by:

1. First clone vidgear:

  # clone the repository and get inside
  git clone https://github.com/abhiTronix/vidgear.git

2. Then, commenting this line, so as to disable enforced cv2.CAP_GSTREAMER backend.

vidgear/vidgear/gears/camgear.py

Line 136 in eb19b9c

backend = cv2.CAP_GSTREAMER

3. Then install vidgear as:

  # install normally
  cd vidgear
  pip install .

4. Running your code again with CAP_FFMPEG backend:

from vidgear.gears.helper import dimensions_to_resolutions
from vidgear.gears import VideoGear

import pafy
import cv2

source = "https://youtu.be/j1GLs_fMn1s"

video = pafy.new(source)
valid_streams = dimensions_to_resolutions(
    [stream.resolution for stream in video.allstreams]
)  # ['144p', '240p', '360p', '480p', '720p', '1080p']

options = {
    "STREAM_RESOLUTION": valid_streams[1],  # '240p'
    "STREAM_PARAMS": {"nocheckcertificate": True},
}

stream = VideoGear(
    source=source, stream_mode=True, backend=cv2.CAP_FFMPEG, **options
).start()

while True:
    frame = stream.read()

    if frame is None:
        break

    cv2.imwrite("frame.jpg", frame)

    break

stream.stop()

[abhiTronix]

@ioangatop Can you confirm your OpenCV version by excuting this command:

python3 -c "import cv2; print(cv2.__version__)"

[ioangatop]

@abhiTronix thanks for the quick response. My OpenCV version is 4.5.1.

I guess there isn't an easier way to change / choose the backend, right?

[abhiTronix]

@abhiTronix thanks for the quick response. My OpenCV version is 4.5.1.

@ioangatop Thanks, I'm looking into OpenCV issues for clue.

I guess there isn't an easier way to change / choose the backend, right?

I can make that simpler, but there's a bug with FFmpeg, which made me to enforce Gstreamer backend in the first place.

[abhiTronix]

@ioangatop OpenCV is crappy in doing VideoIO even with backends. I'm thinking to switch FFmpeg decoding completely with issue #148. But unfortunately, It will going to happen next year due to busy 2021.

I think the best resolution to this problem for now is to use 360p or 720p and then reduce size manually. You can use reducer() for that purpose.

[abhiTronix]

I'm closing this issue for now. I'll look into it in later versions.

[ioangatop]

@abhiTronix thanks! I believe that worse and best in general don't have a problem, but it's good to have an option to fetch frames with a specific resolution, as it is a nice trade-off of speed and quality.

I'm not sure but I hope that in general, only these resolutions have problems, and its not for this specific stream (meaning if I put another stream url, 480p will act properly but not the 720p)

[abhiTronix]

@ioangatop I'm thinking this is to do with GStreamer opencv/opencv#19025 (comment). As we can see errors in logs:

[ WARN:0] global /home/abhishek/Downloads/opencv-master/modules/videoio/src/cap_gstreamer.cpp (898) open OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:0] global /home/abhishek/Downloads/opencv-master/modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=450, duration=-1

[abhiTronix]

and its not for this specific stream (meaning if I put another stream url, 480p will act properly but not the 720p)

@ioangatop These errors/anomalies are only for Youtube livestreams and not general videos. General videos will works out of the box for any resolution.

[abhiTronix]

@ioangatop I think #274 resolves this issue too.

  # install normally
  pip install -U vidgear[core]

Test your code:

from vidgear.gears.helper import dimensions_to_resolutions
from vidgear.gears import VideoGear

import pafy
import cv2

source = "https://youtu.be/j1GLs_fMn1s"

options = {
    "STREAM_RESOLUTION": "240p",
    "STREAM_PARAMS": {"nocheckcertificate": True},
}

stream = VideoGear(
    source=source, stream_mode=True, logging=True, **options
).start()

while True:
    frame = stream.read()

    if frame is None:
        break

    cv2.imwrite("frame.jpg", frame)

    break

stream.stop()

[abhiTronix]

Successfully resolved and merged in commit: dc26c00

Usage docs are available at: https://abhitronix.github.io/vidgear/latest/gears/camgear/usage/#using-camgear-with-youtube-videos