Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warn on unsupported video #7373

Closed
jprochazk opened this issue Sep 9, 2024 · 5 comments · Fixed by #7433
Closed

Warn on unsupported video #7373

jprochazk opened this issue Sep 9, 2024 · 5 comments · Fixed by #7433
Assignees
Labels
🧑‍🎨 Design Requires UX/UI designer input 🎞️ video

Comments

@jprochazk
Copy link
Member

jprochazk commented Sep 9, 2024

When a user loads a video which we can't load for some reason (failed to load, unsupported container format, unsupported codec, etc.), we should report that to them in some way other than a short-lived warning notification.

Some error cases are:

  • Firefox doesn't support WebCodecs outside of Nightly
  • Safari doesn't support AV1
  • Unsupported video container format
  • Unsupported video codec (inside the container)
  • Container parse error
  • Codec configuration error
  • Decode error (WebCodecs)
@gavrelina
Copy link
Member

A ui/ux proposal!

  • highlight the component in the stream that causes an error to improve its discoverability
  • in the error copy itself mention explicitly what's wrong and why and how users can recover from it (i.e. linking to github issue to follow or docs)
  • visually display smth explicitly "broken" instead of the video in the Viewer, like this:
Screenshot 2024-09-09 at 16 00 25

@Wumpf
Copy link
Member

Wumpf commented Sep 9, 2024

👍 makes sense, thank you so much for chiming in here!

A few thoughts:
I think the error text needs to show both when the specific components (like the blob here) is clicked, as well as when the entity containing the broken video is selected. Mostly because it's much more common to click on an entity.

On the streams panel having only components highlight in the error color is probably enough though (and liiikely also too hard otherwise)

As we progress on this we might want to have different error icons and error handling (the loose list in the ticket description will need some structuring), but I'd say for now we should not make much fuss about it.

@gavrelina does this make sense?

@emilk emilk added the 🧑‍🎨 Design Requires UX/UI designer input label Sep 10, 2024
@gavrelina
Copy link
Member

@Wumpf makes a lot of sense!

About the entity msg => it would also then be visible if users click on it via the blueprint, right? that would also largly increase the discoverability! I do worry (slightly) that copying it in few places comes across as "omg! so many errors everywhere, panic!", hence my initial idea was to only show at Blob level and highlight Blob in red so that users would go there to find it. so kinda direct them with color nudging :) what do you think?

And moving forward from here it still makes sense to revise (revive?) the different gradations and make this thing work : #4304

@jprochazk
Copy link
Member Author

From: #7398 (comment)

I don't know if you've experienced this scenario: You're playing a video on the web, and it suddenly fails with an error of some kind. The player just stops, and the controls become inactive, so you can't seek to a different point in the video anymore. But then you reload the page, and seek after the point where the player previously stopped with an error, and the video continues playing back just fine.

I think we could do a lot better than web video players here, by always allowing seek even in the case of an error, and more importantly also highlight which segments are broken on the timeline, so the user knows exactly where to seek next to try and catch a part of the video which can be played back.

This kind of behavior would definitely follow our "handle anything the user throws at it" philosophy

@gavrelina
Copy link
Member

@jprochazk I can already picture how we mark some specific events/part on the timeline where one encounters an "error"!

Wumpf added a commit that referenced this issue Sep 11, 2024
…7398)

### What

* bits and pieces for #7373
    * but far from having that solved

Video decoder forwards now more rich status information upon decode.
This is useful for error handling and - the immediate motivation - to
keep redrawing while we're waiting on pending frames.

Before, when clicking somewhere on the timeline, we wouldn't redraw
until the mouse moves, which for back-seeking (and far forward-seeking)
usually means that we don't get the new frame until the decoder catches
up.

I also took the liberty to move the video re_renderer module around a
bit - it was falsely sorted under `renderer` which are all about
renderers that process draw data and get submitted to a view builder
(which is not the case here!).


https://github.com/user-attachments/assets/623d712b-384b-4ffa-bcbf-3a80885ccef6



### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7398?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7398?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7398)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.
jprochazk pushed a commit that referenced this issue Sep 11, 2024
…7398)

### What

* bits and pieces for #7373
    * but far from having that solved

Video decoder forwards now more rich status information upon decode.
This is useful for error handling and - the immediate motivation - to
keep redrawing while we're waiting on pending frames.

Before, when clicking somewhere on the timeline, we wouldn't redraw
until the mouse moves, which for back-seeking (and far forward-seeking)
usually means that we don't get the new frame until the decoder catches
up.

I also took the liberty to move the video re_renderer module around a
bit - it was falsely sorted under `renderer` which are all about
renderers that process draw data and get submitted to a view builder
(which is not the case here!).


https://github.com/user-attachments/assets/623d712b-384b-4ffa-bcbf-3a80885ccef6



### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7398?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7398?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7398)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.
@Wumpf Wumpf closed this as completed in 30f2877 Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🧑‍🎨 Design Requires UX/UI designer input 🎞️ video
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants