Some videos don't play in webview, but play in Chrome sometimes

[panstromek]

Hi, I've been hunting a strange bug, where some videos (included via html video element) don't play on some Android devices. I'm pretty much desparate, we're getting bad reviews for this, but the problem is somewhere in the system where we have no control over it.

Trying to gather if somebody else encountered similar problem.

Investigation so far:

• The video doesn't load, video element throws "NotSupportedError" with no other information

• It happens on

  • Lenovo IdeaPad Flex chromebook
  • Motorola guamna (moto g play (2021))
  • Hisense U964
  • Motorola G6 (first reported user)
  • ... some other devices that were not reported

• First user reported that this started happening some time around April 12.

  • Some videos that worked before, stopped working for them
  • This happened system wide - both in Chrome and the app (WebView)

• Affects videos system-wide in some way, sometimes

  • On the Chromebook, videos work in Chrome, but don't work in Firefox (Android App) and our app (which uses Android system webview probably)
  • First user reported the problem system wide
  • Other users reported the videos only working in Browser, not in the app

• Affects only some videos

  • All videos are h264, yuv420p, AAC, should be supported by pretty much any device

  • The only pattern that seems to hold is video height

    • From my testing, seems that all videos that are taller than 1088 pixels don't play
      • Tested by generating bunch of videos with same parameters, just different resolutions
      • videos that are shot at landscape but rotated with a metadata flag follow this pattern for width instead

  • I tried to fix this by adding second source tags under the video, with 480p variant (instead of the default 720p)

    • This fixed the problem on the Chromebook and for the first reported user (Motorola G6)
    • After we released new version, we got a new review from the Motorola guamna, reporting this error again.
    • I added 360p variant and I'm waiting for the response

---

Hypothesis

1. System Update

Since this started happening at some point, I suspect some broken system update.

• Android System WebView
  • It wouldn't be the first this one broke something
• Video drivers on Android
  • I believe these phones have HW decoders, so maybe they crash in some specific configurations?
    • Some video codec implementations may have peculiar behaviour on some resolutions, that might be the reason for the weird 1088 pixel limit
• Android Update

2. Battery/CPU optimization

• Some phones degrade performance or disable some features when battery health degrades, might be that
• I've read Android gives a CPU budget to apps, so maybe the app fails to play the video if it's too heavy?
  • In some cases, videos work in Chrome, maybe Chrome is privileged on Android
• All problematic devices so far are low-end, so they might be more prone to these issues

3. Capacitor problem

• Maybe there's something wrong with the WebView or app configuration?
  • This would match the observation that videos work in Chrome but not in other apps sometimes

Has anybody encountered a similar problem?

[ttam]

This definitely seems to be an Android System WebView issue.

I'm seeing very similar things to you in a NativeScript app, and rolling back the WebView from the play store fixes the issue. Obviously not a fix, but the only way we can get certain videos to play again.

[panstromek]

update:

After some more investigation, I found out that webview doesn't actually switch the source if this happens (which usually fixes the issue), because source doesn't throw any error. It seems just that the video is stuck buffering. Here's an investigation report and how I eventually "fixed" it:

Video stuck investigation

Tested on Chromebook in Opera for Android, which also reproduces the bug. (Firefox reports error and correctly switches variant, SA app is not debuggable easily, Chrome works without a bug)

Here's how the html video element behaves when the bug reproduces:

events that still fire:
'timeupdate' (only when seeking or when video is playing for real)
'pause' (seems to be related just to buttons)
'play' (same, just button thingy)
'seeking' = whenever you tap or set currentTime

events that don't seem to fire:
'seeked'

• indicates that it's possible to play the video from the specified position

other events:

• 'loadedmetadata' and 'loadeddata' are incosisent

  • sometimes loadeddata fires, but sometimes not
  • at least one fires every time, we can use it as a trigger for bug detection

• 'suspend' event seem to be firing somewhat randomly, probably not reliable source of truth

• 'waiting' might be okay as an initial trigger, but fires often

• 'canplay' firing even when bug happens and video can't play

• 'stalled' would match semantically, but doesn't fire when the bug reproduced

---

properties that match

• readyState is stuck on 1 (LOADED_METADATA) when the bug happens, could be used

other properties on video that are not a match

• networkState seems to be 1 (IDLE) all the time (no matter if the video is playing or not)
• error is null (in our problematic case)
• paused is false

---

possible solutions

• seek to 0 everytime we load the video

  • see if we get seeked events
  • if we don't get 'seeked' event in X seconds, reorder variants to prefer no-HD
  • Pros
    • relatively simple, single thing to check
  • Concerns:
    • can seek to 0 ever be problematic?
    • what if there's just unusual delay (slow network)?

• watch time updates

  • if we don't get "enough" time updates, reorder variants to prefer no-HD

  • Pros:

    • should more accurately detect that video actually plays (lot of time updates)

  • Concerns:

    • "enough" treshlod might be flaky
      • timeUpdate fires when user is seeking on timeline, we have to account for that
    • more book keeping, we have to measure some statistics

  • "enough" definition options

    • just count how many time updates we get
      • maybe subtract seekes
      • dragging the seek generates a ton of seeking events :/

  • alternative to timeUpdate would be checking the currentTime manually with RAF

    • similar but more reliable, timeUpdates are not fired on every frame

---

Conclusion

Reliable state to look at

• seeked event not firing
• video stuck in readyState=1 when it should be playing
• frequency of time updates

We will go with the 1st solution option for now

• detecting seeked event
• if we don't 'seeked' in 2 seconds, we switch variants

[derWebdesigner]

@panstromek Did you solve this issue somehow? I am currently experiencing a very similar issue and it feels like the videos randomly get stuck somewhere because sometimes they load, sometimes they don't. As long as I am running the app as a non-compiled production version it's always working but as soon as I run ionic capacitor run android --prod it feels like total randomness.

I think I solved the issue for myself by setting preload="{{ isAndroid ? 'none' : 'auto' }}". As soon as Android isn't preloading the videos anymore it's working correctly, I still have to keep it for iOS though because with preload="none" you get a short graphical issue that simply looks ugly.

[panstromek]

I only solved it in the way I described above - try to play the video, wait for seeked event, if we don't get seeked event in 1s, try to switch programmaticaly to different source (with lower dimensions).

Your issue seems different, though. The issue described here is completely predictable on the affected devices and browsers - all videos that have height over 1088 don't play, that's it. Sometimes it could temporarily help to reinstall or downgrade or upgrade Android System Webview, but the issue would soon re-appear, Android is probably juggling those versions at runtime somehow.

Your problem is probably preload="auto", you should never use that setting. I experienced something similar, too. preload="auto" means that the browser has to download every video file in its entirety, so it quickly maxes out resources in the phone, which results in some videos just going black or not playing correctly and the whole UI lagging a lot. You should either use "none", "metadata" or omit the attribute completely, that will let the browser choose effectively how much of the video it should download.

[derWebdesigner]

@panstromek Thank you very much for your additional hints, I highly appreciate it and will read more into the preload setting. I was only using it because it's the default but it's definitely interesting how this behaves and why it even behaves differently when it's in production mode. I hope my answer helps others who suffer from the same issue I did because it cost me an entire day to figure this out...