Video decode performance (Bezels/Overlays) etc...

[aaciii]

Have discovered that when the Bezel feature is enabled, video and audio are out of sync (noticeable on longer video clips). Video appears to playback more slowly with the Bezel enabled whereas audio is unaffected.

Is the video playback tuneable?

[DirtBagXon]

Video/audio sync issues are almost invariably resource exhaustion.

You don't mention your hardware, or the video resolutions you are seeing the issue on, but I suspect your hardware is struggling. There is no means to tune video playback as it will always be driving to be frame accurate. Re-encoding and lowering the video resolutions with ffmpeg would probably be the usual cure.

The bezel is just loaded as a texture which is copied over the game video/overlay at the final render stage. So the GPU hardware will be taking the grunt here.

[aaciii]

It is an AtGames Legends Ultimate arcade unit; the chipset is a Rockchip 3328. Running it as 1280x720 (the Dragons Lair source I'm using is 960x720).

The file format of the source is MPEG2 video and Ogg Vorbis audio. Appears to have been extracted from DVD (filenames have "VOB" in them). Is there a video format that would be more efficient for Hypseus to decode?

[DirtBagXon]

"Rockchip 3328" are $10 RPi3 SBC equivalents - I don't know the hardware - Google told me that.

I would suggest sticking with original Daphne 640(720)x480 Laserdisc rips on hardware of that spec, especially if you are loading on bezels.

The files i have seen containing vob - e.g. dls21.vob.m2v are

Width                                    : 1 440 pixels
Height                                   : 1 080 pixels

hypseus can handle 4k MPEG2 provided the hardware it's running on can provide the resources, the RPi3 is edge case on more than original SD content (even with overclocking), I would have to assume the bezels have just pushed it over the edge.

[DjDiabolik]

This my consideration does not really deal with the bezels (what should be the bars on the sides of the video played in 4:3 on a 16:9 monitor exactly?) but in general whit video decode/reproduction in general.

I think I can join this discussion to make my own consideration and tell you my experience:
@DirtBagXon on my case i'm a totally different hardware.
I'm installing UBUNTU 22.04.3 LTS on a Intel NUC NUC7PJYHN (buyed from Amazon) that have which has a 480GB SSD and 8GB of DDR4 RAM.
So, in this hardware i have 'installed' the current retropie 'ecosystem' and i have add (build from latest source) the current hypseus.
After that i have add DL2e at games (simply copyed the DL2e from singe 2 branch and copy the m2v and ogg converted from mp4 using ffmpeg and the suggested options).
Said this, apparently all works correctly but in some specific point of games i noticed cases of out-of-sync.... for example on a specific point.
At the very end of level 1 (after the scene 3) when the big snake is hit by a bludgeon on the head. In this specific case i can listen clearly the audio track goes completely out-of-sync.

Trust me...... I've tried everything these past few days! Try to re-encode the video whit a lower bitrate (used -an -b:v 2500k obviously obtaining a smaller file but with a worse visual quality) but nothings apparently can help to remove this out-of-sync in this specific scene.
I have also tryed to re-econde so many time the ogg track!! whit increasing the value each time -ss 00:00:00.330 suggested but also here. Never found a correct solution. In this specific point out-of-sync.

After several tests with different combinations of m2v and ogg by google i have found a forum reply by someone speaked about try change the OVERALL resolution in service menù. So. I HAVE TRYED THAT AND BAAAM!!!
It's the solution!!!! YEAH!! Immediatelly also on DEMO if i set on "Performance":
CONFIGURATION - > FULL
But OVERLAY RES - > MINIMAL (font size to -2 or all it's too big)
Basically I set the options like this. I took you a screenshot of the screen directly:

When i exit from 'service' menù BAAAM..... I immediately notice the video as if it is playing at a faster speed!!
Also the same gameplay........ all it's everything seems to be going much better..... too much much more smoothly i don't thinks if i can explain.

So simply to set the OVERLAY RES to mininal instead the 'default' HI it's resolve all my trouble also if i use m2v whit a 6000k in bitrate. Also all out-of-sync disappears completely.

Overlay should be the part where it shows the exact score and remaining lives ? it may be that by simply scaling the definition of the latter can help to remove all sinthom of out-of-sync ?

[DirtBagXon]

"Video/audio sync issues are almost invariably resource exhaustion." - Same answer applies

Yes, altering the overlay size will vastly improve the situation. The size of the overlay is what consumes resources. Most of the original Daphne and Singe 1 games run overlays fixed at 360x240 (like many classic arcade games) - Singe 2 overlays, on 1080p video, will try and run at 1920x1080 or 960x720 - so you can see the jump.

You are trying to run full overlay Singe 2 games on a mid-range NUC using a Intel® UHD Graphics 605.

If you see sync issues, look at the CPU status in linux top or equivalent.

This is currently why the RPi4 uses 720p video and the minimal overlay on Singe 2 games for the optimum experience.

There are discussions on implementing an improvement in this rendering, that should help these smaller devices display SDL textures more efficiently, but this is some way off at the moment, if it comes to fruition.

[DjDiabolik]

OMG very interesting.
in one of my test, i have tryed to reduce the bitrate of m2v but mantain the same resolution at 1440x1080.
It's not a big trouble......... i will also try to reduce the resolution from source DL2e.mp4 to obtain a good quality a 720p.

I thinks i can only try adapt my ffmpeg command........ for obtain this you suggest two different ffmpeg commands here:

ffmpeg -i game.mp4 -an -qscale:v 4 -b:v 6000k -vf scale=1280:720 -codec:v mpeg2video game.m2v

ffmpeg -i game.mp4 -an -qscale:v 4 -b:v 6000k -vf tpad=stop_mode=clone:stop_duration=2,scale=1280:720 -codec:v mpeg2video fgame.m2v

what is the difference between the two command lines?

For another example here:
#60 (comment)
You also suggest to use 960x540............ i try this for a purpose test because encoding doesn't even take 10 minutes anyway :)

[DirtBagXon]

tpad=stop_mode=clone:stop_duration=2 - what this does is pad the end of the m2v with a few extra frames.

This is because some game developers like to use the very last frames of the video and this can sometimes be problematic.

There is no harm in doing this in every encode, it will simply add a few more frames to the end of the video.

[DjDiabolik]

Immediatelly i can try....... this is how i can obtain........ a newest DL2e.m2v at 960x540.......... on my Windows PC:

His mediainfo it's:

General
Complete name               : D:\MeGUI\tools\ffmpeg\Dl2e_540.m2v
Format                      : MPEG Video
Format version              : Version 2
File size                   : 763 MiB
Duration                    : 29 min 40 s
Overall bit rate mode       : Variable
Overall bit rate            : 3 596 kb/s
Frame rate                  : 23.976 FPS

Video
Format                      : MPEG Video
Format version              : Version 2
Format profile              : Main@High 1440
Format settings, BVOP       : No
Format settings, Matrix     : Default
Format settings, GOP        : Variable
Duration                    : 29 min 40 s
Bit rate mode               : Variable
Bit rate                    : 3 596 kb/s
Width                       : 960 pixels
Height                      : 540 pixels
Display aspect ratio        : 4:3
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Compression mode            : Lossy
Bits/(Pixel*Frame)          : 0.289
Time code of first frame    : 00:00:00:00
GOP, Open/Closed            : Open
GOP, Open/Closed of first f : Closed
Stream size                 : 763 MiB (100%)
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709

Used that commands:
ffmpeg -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=960:540 -an -qscale:v 4 -c:v mpeg2video Dl2e_540.m2v

Now move this file on my Intel NUC and try the result immediatelly.........

I hope I'm not messing around too much....

[DirtBagXon]

In regard to the scale sizes, just ensure you keep the aspect ratio correct.

This is a useful site: https://andrew.hedges.name/experiments/aspect_ratio/

[DjDiabolik]

In regard to the scale sizes, just ensure you keep the aspect ratio correct.

This is a useful site: https://andrew.hedges.name/experiments/aspect_ratio/
oh yes... apparently ffmpeg it's it's smart enough on its own to figure out that the source video is 4:3 and respect that in the output m2v.

So i have tryed right now the 540p video and respect the 1080p the difference in quality is noticeable maybe,i thinks, because my Intel NUC it's attached to my LG TV and it's setted at desktop at 1080p so i'm playing a 540p file on a 1080p resolution and so it loses quality.

I have to try to force to use a biggest bitrate on video encoding to preserve a good re-encoding.. i have tryed that:
ffmpeg -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=960:540 -an -qscale:v 4 -b:v 6000k -c:v mpeg2video Dl2e_540.m2v
But it's not works. Maybe i need to remove the quality scale 4 ? I need to try....... or i can try to raise video at 720p....

EDIT 1st tentative
Used that commands:
ffmpeg -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=960:540 -an -b:v 6000k -c:v mpeg2video Dl2e_540_hb.m2v

'hb' stands for HIGH BITRATE.......lol:

General
Complete name               : D:\MeGUI\tools\ffmpeg\Dl2e_540_hb.m2v
Format                      : MPEG Video
Format version              : Version 2
File size                   : 1.19 GiB
Duration                    : 29 min 40 s
Overall bit rate mode       : Variable
Overall bit rate            : 5 734 kb/s
Frame rate                  : 23.976 FPS

Video
Format                      : MPEG Video
Format version              : Version 2
Format profile              : Main@High 1440
Format settings, BVOP       : No
Format settings, Matrix     : Default
Format settings, GOP        : Variable
Duration                    : 29 min 40 s
Bit rate mode               : Variable
Bit rate                    : 5 734 kb/s
Width                       : 960 pixels
Height                      : 540 pixels
Display aspect ratio        : 4:3
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Compression mode            : Lossy
Bits/(Pixel*Frame)          : 0.461
Time code of first frame    : 00:00:00:00
GOP, Open/Closed            : Open
GOP, Open/Closed of first f : Closed
Stream size                 : 1.19 GiB (100%)
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709

Test immediatelly after this alternative m2v how it's 'played' on my intel nuc hardware.

EDIT 2nd Tentative
Tryed to obtain a 720p whit this commands:
ffmpeg -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=1280:720 -an -qscale:v 1 -b:v 6000k -c:v mpeg2video Dl2e_720_hb.m2v

Specify the -qscale:v 1 it's not respect the -b:v 6000k infact i abort this encoding after about 10 minutes of video whit a output filesize about 2GB.
I need to found the better and good value 'for my viewing pleasure' (lol Dl2e appears on my tv) and also for my intel nuc hardware.
Now try to re-encode whitout -qscale but only whit force bitrate commands...

3rd test
Lol at 720p problably i have obtain a file larger than necessary:

General
Complete name               : D:\MeGUI\tools\ffmpeg\Dl2e_720_hb.m2v
Format                      : MPEG Video
Format version              : Version 2
File size                   : 1.55 GiB
Duration                    : 29 min 40 s
Overall bit rate mode       : Variable
Overall bit rate            : 7 465 kb/s
Frame rate                  : 23.976 FPS

Video
Format                      : MPEG Video
Format version              : Version 2
Format profile              : Main@High 1440
Format settings, BVOP       : No
Format settings, Matrix     : Default
Format settings, GOP        : Variable
Duration                    : 29 min 40 s
Bit rate mode               : Variable
Bit rate                    : 7 465 kb/s
Width                       : 1 280 pixels
Height                      : 720 pixels
Display aspect ratio        : 4:3
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Compression mode            : Lossy
Bits/(Pixel*Frame)          : 0.338
Time code of first frame    : 00:00:00:00
GOP, Open/Closed            : Open
GOP, Open/Closed of first f : Closed
Stream size                 : 1.55 GiB (100%)
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709

I need simply to found the right compromise. Do you think it's useless to use such a high bitrate?

[DirtBagXon]

You have to find what works for you. Try -scanlines as sometimes this can sharpen the video to the naked eye.

Your TV upscaling is probably the issue here too from what you describe.

[DjDiabolik]

OMG!!! In all previous TEST i always SET the wrong resolution on output m2v!!!! OMG.... 1440 x 1080 the source mp4. The correct 4:3 for video at 540p it's 720x540 !!!!!!

OMG.... and you pointed it out to me from the beginning!!! LOL...... now need to rencode all my test video :)

as already mentioned, fortunately the encoding procedure takes more or less 10 minutes.....

[DjDiabolik]

You have to find what works for you. Try -scanlines as sometimes this can sharpen the video to the naked eye.

Your TV upscaling is probably the issue here too from what you describe.

Lol... I also think the problem is there. But I also think that if I feed it a, shall we say, better file it should help in that regard.
Anyway now need to re-encoding all this testing video whit the newest and correct resolution....... whit this:
ffmpeg -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=720:540 -an -b:v 7500k -c:v mpeg2video Dl2e_540_hb_alt.m2v
Obtained this:

General
Complete name               : D:\MeGUI\tools\ffmpeg\Dl2e_540_hb_alt.m2v
Format                      : MPEG Video
Format version              : Version 2
File size                   : 1.11 GiB
Duration                    : 29 min 40 s
Overall bit rate mode       : Variable
Overall bit rate            : 5 359 kb/s
Frame rate                  : 23.976 FPS

Video
Format                      : MPEG Video
Format version              : Version 2
Format profile              : Main@Main
Format settings, BVOP       : No
Format settings, Matrix     : Default
Format settings, GOP        : Variable
Duration                    : 29 min 40 s
Bit rate mode               : Variable
Bit rate                    : 5 359 kb/s
Width                       : 720 pixels
Height                      : 540 pixels
Display aspect ratio        : 4:3
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Compression mode            : Lossy
Bits/(Pixel*Frame)          : 0.575
Time code of first frame    : 00:00:00:00
GOP, Open/Closed            : Open
GOP, Open/Closed of first f : Closed
Stream size                 : 1.11 GiB (100%)
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709

Don't understand because i specify set bitrate to 7500k ffmpeg using a bitrate variable at 5359k.

Maybe because ffmpeg tends to waste bitrate if it doesn't deem it necessary?

[DirtBagXon]

Try removing -b:v argument and play with -qscale - 4 the default, is quite high - See docs.

https://trac.ffmpeg.org/wiki/Encode/MPEG-4#VariableBitRatewith-qscale

You may have tried this already.....

[DjDiabolik]

Try removing -b:v argument and play with -qscale - 4 the default, is quite high - See docs.

https://trac.ffmpeg.org/wiki/Encode/MPEG-4#VariableBitRatewith-qscale

You may have tried this already.....

So, whit the corrected Aspect Ratio and 'new' m2v (the last mediainfo at 720x540) the quality it's good.
if i get really close to my tv maybe you notice some defects but if you move away from the tv everything seems perfect.... lol... my 5 year old niece would play 30/40cm from the TV anyway.

Now need to re-encode the 720p version whit correct resolution.... 960x720 insted previously used 1280x720.
Also tryed to use -qscale 4.......... to obtain this.

EDIT
I also need to see if exist a method to create two different games on retropie/emulation...... a way to create a duplicate of the same game.
only in this way can I try the two videos in a row and possibly notice the real differences.

1st TEST at 720p
Whit command:
ffmpeg -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=960:720 -an -qscale:v 4 -c:v mpeg2video Dl2e_720_alt.m2v
I obtain this:

General
Complete name               : D:\MeGUI\tools\ffmpeg\Dl2e_720_alt.m2v
Format                      : MPEG Video
Format version              : Version 2
File size                   : 899 MiB
Duration                    : 29 min 40 s
Overall bit rate mode       : Variable
Overall bit rate            : 4 235 kb/s
Frame rate                  : 23.976 FPS

Video
Format                      : MPEG Video
Format version              : Version 2
Format profile              : Main@High 1440
Format settings, BVOP       : No
Format settings, Matrix     : Default
Format settings, GOP        : Variable
Duration                    : 29 min 40 s
Bit rate mode               : Variable
Bit rate                    : 4 235 kb/s
Width                       : 960 pixels
Height                      : 720 pixels
Display aspect ratio        : 4:3
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Compression mode            : Lossy
Bits/(Pixel*Frame)          : 0.256
Time code of first frame    : 00:00:00:00
GOP, Open/Closed            : Open
GOP, Open/Closed of first f : Closed
Stream size                 : 899 MiB (100%)
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709

bitrate used 4235k also the size it's very good about 920MB for absurd a little size respect the 540p whit -b:v 7500k video.

I can also try to use -b:v 7500v for 720p video..... anyway now need to delete all video tested whit wrong resolution.

2nd TEST whit 720p
Using:
ffmpeg -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=960:720 -an -b:v 7500k -c:v mpeg2video Dl2e_720_hb.m2v

obtain this:

General
Complete name               : D:\MeGUI\tools\ffmpeg\Dl2e_720_hb.m2v
Format                      : MPEG Video
Format version              : Version 2
File size                   : 1.45 GiB
Duration                    : 29 min 40 s
Overall bit rate mode       : Variable
Overall bit rate            : 6 983 kb/s
Frame rate                  : 23.976 FPS

Video
Format                      : MPEG Video
Format version              : Version 2
Format profile              : Main@High 1440
Format settings, BVOP       : No
Format settings, Matrix     : Default
Format settings, GOP        : Variable
Duration                    : 29 min 40 s
Bit rate mode               : Variable
Bit rate                    : 6 983 kb/s
Width                       : 960 pixels
Height                      : 720 pixels
Display aspect ratio        : 4:3
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Compression mode            : Lossy
Bits/(Pixel*Frame)          : 0.421
Time code of first frame    : 00:00:00:00
GOP, Open/Closed            : Open
GOP, Open/Closed of first f : Closed
Stream size                 : 1.45 GiB (100%)
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709

Regarding the official ffmpeg manual the -b:v xxxxxk it's correct to use whit a two pass encoding to obtain a better result.

I can also try to encode whit two pass and bitrate setted to 6000k to see how i can obtain........

[DjDiabolik]

Sorry me..... but at this point it's simply a my personal purpose test but... until now... i also want to upgrade about all tentative.

VERY IMPORTANT THINGS!!! TO RESOLVE MY PRINCIPAL ISSUE IT'S ALL CLEARLY... Apparently in my Hardware if i use every at 720p or every video at 540p resolve completely all out-of-sync trouble.
Whit lover resolution of 1440x1080 for m2v i can, RIGHT NOW, use the 'OVERLAY RES' to HI whitout any issue or simply can set 'Configuration' to FULL whit all other default setting.

At this point it's only a purpose test.......... whit all my latest test this time i finally obtain problably my final choosing m2v.
Using the TWO PASS ENCODING........ for doing it, i have create a .bat files whit this commands inside:
ffmpeg -y -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=960:720 -an -b:v 6000k -pass 1 -c:v mpeg2video -f null NUL && ffmpeg -i DL2e.mp4 -vf tpad=stop_mode=clone:stop_duration=2,scale=960:720 -an -b:v 6000k -pass 2 -c:v mpeg2video Dl2e_720_2p_6000k.m2v

In this mode after first pass you can obtain a log files... i file called ffmpeg2pass-0.log.
Whit pass two you can finally obtain the output files....... in my case i have obtain this:

General
Complete name               : D:\MeGUI\tools\ffmpeg\Dl2e_720_2p_6000k.m2v
Format                      : MPEG Video
Format version              : Version 2
File size                   : 1.24 GiB
Duration                    : 29 min 40 s
Overall bit rate mode       : Variable
Overall bit rate            : 6 000 kb/s
Frame rate                  : 23.976 FPS

Video
Format                      : MPEG Video
Format version              : Version 2
Format profile              : Main@High 1440
Format settings, BVOP       : No
Format settings, Matrix     : Default
Format settings, GOP        : Variable
Duration                    : 29 min 40 s
Bit rate mode               : Variable
Bit rate                    : 6 000 kb/s
Width                       : 960 pixels
Height                      : 720 pixels
Display aspect ratio        : 4:3
Frame rate                  : 23.976 (24000/1001) FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Compression mode            : Lossy
Bits/(Pixel*Frame)          : 0.362
Time code of first frame    : 00:00:00:00
GOP, Open/Closed            : Open
GOP, Open/Closed of first f : Closed
Stream size                 : 1.24 GiB (100%)
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709

I don't have idea if using the TWO PASS encoding it's helped to be obtain this result but look at right now....... the 6000k choosing at commands it's right now fully respected.

Now only need to also test this addictional file on my Intel NUC........

I'm trying to figure out if it's possible to create multi folders for the same game maybe using the same .ogg file for the audio..... but I think not.

[DjDiabolik]

Ohhhh... finally i got also write directly from UBUNTU (after i have login my account also on ubuntu on nuc).
From here...... i can also confirm:
It's possibly to create a multiple folder for same games on retropie/emulationstation.
It's simply need to create for example... DL2e_720p.daphne and DL2e_540p.daphne
That solution it's work but you need also to modify all files inside like the main .singe in my case it's named DL2e_540p.singe and DL2e_540p.txt
You also need to modify the files .singe internally using a normal text editor... for example in DL2e_540p.txt i need to edit this line like this:
MYDIR = BASEDIR .."/".. "DL2e_540p" -- This is the directory where your game is

If here you leave out Dl2e it's not works... i thinks you need to declare correct folder....
In my case from framefile .txt i can also load all m2v/ogg directly from the DL2e.daphne simply load the framefile like this:
../DL2e.daphne/Video/

Now i can ask one things... now i need to put so many copy of same .ogg files.. but problably i can also create a main .ogg and some link instead to create duplicate of same file .ogg.

So after this........ i can also put this SCREENSHOT from my Intel NUC...... this is the screenshot of my current 540p version of DL2e:

And finally for now a screenshot of 720p of DL2e:

Trust me... it's very difficult to identify the real difference.
I also know.... the two different screenshot it's not possibly at same frame but it's for a general consideration... look the general quality.
The real difference resolution apparently it's not noticeable like also the totally different bitrate (the 720p use a 6000k whit 2 pass encoding and 540p about 7500k).