Generated diff image does not show all differences

[simonwalter]

We have a lot of red/reddish elements in the UI of our app and we realised that changes in those elements are not reflected in the diff images of the Shot verification report.

ℹ️ This test result shows a screenshot test in which an orange button has been moved slightly downwards. The change is correctly identified by Shot, but it's not clear from the diff image.

Expected behaviour

All differences between the original and the new screenshot are visible in the diff images in the verification report.

Actual behaviour

Differences of some elements are not visible in the diff images in the verification report: This mostly happens with red elements. But in the example below it also happens for green and blue elements when they are placed on black background.

Steps to reproduce

I created a test with a simple UI to reproduce the problem:

package example.test

import android.graphics.Color
import android.view.Gravity
import android.widget.LinearLayout
import android.widget.TextView
import androidx.test.platform.app.InstrumentationRegistry
import com.karumi.shot.ScreenshotTest
import org.junit.Test

class ColorScreenshotTest : ScreenshotTest {

    private val textViewGravity = Gravity.LEFT  // change to Gravity.RIGHT for the verification run

    @Test
    fun colorTestBlack() {
        actualTest(backgroundColor = Color.BLACK)
    }

    @Test
    fun colorTestWhite() {
        actualTest(backgroundColor = Color.WHITE)
    }

    private fun actualTest(backgroundColor: Int) {
        val context = InstrumentationRegistry.getInstrumentation().targetContext

        val view = LinearLayout(context).apply {
            orientation = LinearLayout.VERTICAL
            setBackgroundColor(backgroundColor)

            addView(
                TextView(context).apply {
                    textSize = 30f
                    text = "Red text"
                    setTextColor(Color.RED)

                    gravity = textViewGravity
                }
            )

            addView(
                TextView(context).apply {
                    textSize = 30f
                    text = "Green text"
                    setTextColor(Color.GREEN)

                    gravity = textViewGravity
                }
            )

            addView(
                TextView(context).apply {
                    textSize = 30f
                    text = "Blue text"
                    setTextColor(Color.BLUE)

                    gravity = textViewGravity
                }
            )
        }

        compareScreenshot(
            view = view,
            heightInPx = 500,
            widthInPx = 500
        )
    }
}

Version of the library

5.14.1

More information

I guess that this is related to the RedComposite from the scrimage library which is used to merge the original and the new screenshot to create the diff image.

Shot/core/src/main/scala/com/karumi/shot/screenshots/ScreenshotsDiffGenerator.scala

Line 37 in 4f609da

val diff = newImage.composite(new RedComposite(1d), originalImage)