Video playback and Image questions

[Glazzes]

I do have a couple of questions about different topics, so, I rather ask them all at once than creating a discussion for each one, Thank you in advance for your taking your time to reply to my questions.

Image

1. We can currently take a screenshot of a skia canvas as a ByteArray or a Base64 string. Could we expect the image to be saved to the device's hard drive directly?
2. Is it possible to add headers to useImage? For instance, const image = useImage(url, {Authorization: token}), for protected images it forces you to download it before hand with a library like react-native-file-system or expo-file-system
3. Gifs do not "move" in skia canvas, is there a reason to it? (I'm not sure if it's my phone)

Video

1. If Flutter renders everything through Skia. Would it be possible for react native to play video within an skia canvas? For instance something like this https://gl-react-cookbook.surge.sh/video?menu=true
2. If we could play video, would it be possible to apply a shader to it and save every frame with the shader applied to it (save the whole new video to hard drive) ? For instance, something similar to react-native-vision-camera's new features

[wcandillon]

@Glazzes we know support offscreen rendering which might be of interest to you. We give some hints on how to use it at #1359

Using Videos as a Skia texture would be really cool. If you have a native implementation that creates an OpenGL (for Android) and Metal (for iOS) texture which is updated with video frame, we can probably find an extension point where we can load it as a Skia texture.

[Glazzes]

Thanks for replying and excuse me I don't know much about shaders, when using avideo as a texture, could you seek for a specific time? I assume rendering engines can not play audio, so one would need to play video's audio with a different library

[wcandillon]

We published video support in 1.3

[mrEuler]

Can I somehow seek frame by frame (scrubbing) the video in Android? Because if yes, that is huge pro to use RNSkia instead of RNVideo!
Thanks!

[wcandillon]

I will ship it early next week

…

On Thu 23 May 2024 at 12:00, OleksiiMaksymov ***@***.***> wrote: Can I somehow seek frame by frame (scrubbing) the video in Android? Because if yes, that is huge pro to use RNSkia instead of RNVideo! Thanks! — Reply to this email directly, view it on GitHub <#1326 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKXVSFP465XNQBM7Z7I3TZDW42ZAVCNFSM6AAAAAAUPNNN7GVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TKMZSGU4TI> . You are receiving this because you modified the open/close state.Message ID: <Shopify/react-native-skia/repo-discussions/1326/comments/9532594@ github.com>

[mrEuler]

@wcandillon Just in case tried already video feature today, look good! Only few things find a bit off:

1. some video are rotated incorrectly (portrait becomes landscape)
2. incorrectly FPS detection
3. I changed useVideo hook to get the "video" variable outside of that hook, called .seek(t:ms) function with custom scrubber -- only few frames were changed in 10 seconds video. I've also put 'true' value in if statement, where video.nextImage(); is executed.

Anyway waiting for the new version! If scrubbing on Android will work, this going to be killer feature to use this, instead of RNVideo!

Thanks!

[mrEuler]

Also quick check, if I make with ffmpeg video all frames being keyframes -- I can scrub back and forward.

[mrEuler]

Moving this to separate issue:
#2442