Composable as input for androidViewFilterRender

[maxgrrs]

Hey Pedro,
I want to create different graphic overlays for some small games.
As first step I just thought about a basic result overlay like used in football (two teams with a score each, black text on a white rectangle background).

As far as I understand this list of filter are available out of the box with your library and they can be used more or less in this way.

Looking through this list I thought of androidViewFilterRender as the best option to create that overlay.
I would love to use composables to create those views and tried to use ComposeView as introduced in this android developer guide.

I created an easy example and wanted to integrate it into a CameraPreview Composable:

@Composable
fun CameraPreview(
    modifier: Modifier = Modifier,
    cameraInstance: RtmpCamera2
) {
    Surface(
        modifier = modifier,
        color = MaterialTheme.colorScheme.background
    ) {
        AndroidView(factory = {
            OpenGlView(it).apply {
                val view = this
                view.setAspectRatioMode(AspectRatioMode.Adjust)
                view.holder.addCallback(object : SurfaceHolder.Callback {
                    override fun surfaceCreated(p0: SurfaceHolder) {

                    }

                    override fun surfaceChanged(p0: SurfaceHolder, p1: Int, p2: Int, p3: Int) {
                        cameraInstance.replaceView(view) //attach preview to camera
                        cameraInstance.startPreview() //open camera

                        setViewToStream(cameraInstance, it)
                    }

                    override fun surfaceDestroyed(p0: SurfaceHolder) {
                        cameraInstance.stopPreview() //close camera
                    }
                })
            }
        }, modifier = Modifier.fillMaxSize())
    }
}

private fun setViewToStream(cameraInstance: RtmpCamera2, context: Context) {
    val androidViewFilterRender = AndroidViewFilterRender()
    val composeView = ComposeView(context).apply {
        setContent {
            Surface(
                modifier = Modifier.fillMaxSize(),
                color = MaterialTheme.colorScheme.background
            ) {
                Text("Test")
            }
        }
    }

    androidViewFilterRender.view  = composeView
    cameraInstance.glInterface.setFilter(androidViewFilterRender)
}

But running the application always ends in an IllegalStateException:

java.lang.IllegalStateException: Cannot locate windowRecomposer; View androidx.compose.ui.platform.ComposeView{6ef2b4a V.E...... ......I. 0,0-0,0} is not attached to a window

Searching the web I was not able to find any solution that applies here. Maybe you have any idea?
If not I guess doing it the old xml based way will work as well.

Or do you recommend using other filters for the use case I specified?

Thanks, Max

[pedroSG94]

Hello,

AndroidViewFilterRender require that the view is inflated before set the filter because it is using the canvas render only available while it is inflated.
In your case you are not inflating the composable so you have this error. To inflate the composable you need add the composable to your screen making it visible.

If you want use the view invisible (in background) you can try inflate the view like this:
https://stackoverflow.com/questions/2335813/how-to-inflate-one-view-with-a-layout
And then you can add the composable. It is used for XML so, I suggest you test with a XML and then use composable inside the XML:
https://stackoverflow.com/questions/65648904/android-jetpack-compose-and-xml-in-activity

[maxgrrs]

Hey Pedro,

thanks again for your fast response.
I tried to follow allong your advices and decided to go for AndroidViewBinding in the end to do the inflating, also because I had problems with other approaches as Im not acting from the MainActivity:

@Composable
fun CameraPreview2(cameraInstance: RtmpCamera2, modifier : Modifier = Modifier) {
    Surface(
        modifier = modifier,
        color = MaterialTheme.colorScheme.background
    ) {
        AndroidViewBinding(XmlRootBinding::inflate)  {
            val rootXml = this.rootXml

            val cameraView = rootXml.findViewById<OpenGlView>(R.id.surfaceView)
            cameraView.setAspectRatioMode(AspectRatioMode.Adjust)
            cameraView.holder.addCallback(object : SurfaceHolder.Callback {
                override fun surfaceCreated(p0: SurfaceHolder) {

                }

                override fun surfaceChanged(p0: SurfaceHolder, p1: Int, p2: Int, p3: Int) {
                    cameraInstance.replaceView(cameraView) //attach preview to camera
                    cameraInstance.startPreview(1280, 720) //open camera

                    val overlayView = rootXml.findViewById<ComposeView>(R.id.composable_template)
                    overlayView.setContent {
                        Text("Test", color = Color.Red)
                    }

                    val androidViewFilterRender = AndroidViewFilterRender()
                    androidViewFilterRender.view = overlayView
                    cameraInstance.glInterface.setFilter(androidViewFilterRender)
                }

                override fun surfaceDestroyed(p0: SurfaceHolder) {
                    cameraInstance.stopPreview() //close camera
                }
            })
        }
    }
}

This is the related xml layout:

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/root_xml"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <com.pedro.library.view.OpenGlView
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:id="@+id/surfaceView"
    />

    <androidx.compose.ui.platform.ComposeView
        android:id="@+id/composable_template"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
    />
</RelativeLayout>

The result is not completly as expected, as two instances of the composable show up:

As well as an endless error in the logcat:

updateAcquireFence: Did not find frame.

I guess I somehow messed things up, but Im not able to figure out what.
Can you maybe point me in the right direction again?

Thanks, Max

[maxgrrs]

Hey @pedroSG94,

you maybe have some time to reply to my request?
Would be super nice.

Regards, Max

[pedroSG94]

Hello,

This is the expected result. The filter copy a view and set it to the stream so all is working as expected. You have the original view and the copy in the stream.
You can try hide the original component using other layout over the component. This way you only will have the component in the stream because the original is hide under the layout.
Currently, you are not inflating the view using the way I suggested you using an inflate that is also other way to solve it.