5 Color processing tutorial

This part describes the process of transduction and color processing, which involves the areas Retina, LGN, V1, and V4.

Retina

RetinaProcess

The area of Retina is in charge of transforming an RGB image to LMS activations. LMS means the activations of the Low, Medium, and Short frequency cones. In the system, the result is performed by multiplying each vector RGB of the original image by a 3x3 matrix that represents the conversion from RGB->XYZ->LMS. However, the matrix conventions matrices need to be separate for better understanding. The matrix is:

$$\begin{bmatrix} L\\ M\\ S \end{bmatrix}=\begin{bmatrix} 0.4275& 0.4990 & 0.0472\\ 0.2206& 0.7030 & 0.0918\\ 0.0270& 0.0707& 0.9911 \end{bmatrix} \begin{bmatrix} R\\ G\\ B \end{bmatrix}$$

The LMS activations can be viewed in the left part of the visualizer, where the images are in grayscale, because the matrices are of one channel:

Lateral Geniculate Nucleus

LGNSimpleOponnentCells

In this stage, LMS activations are converted into the DKL color space. Where the activation D,K,L are obtained performing a difference of the activations of the LMS matrices as follows:

• D = L-M
• K = S-(L+M)
• L = L+M

The DKL color space can be represented in a 3D image:

In the implementation, this operation is not a simple summation, a convolution is made in the LMS activation. There are two methods we can choose to perform the operation.

Method 1 is to perform convolutions with normalized gaussian filters (Filter where the sum of all pixels is 1) and then summing using a proportion as follows.

$D=\alpha_{LMM}(L * G( \sigma_1 ))-\beta_{LMM}(M * G( \sigma_2 ))$

$K=\alpha_{SMLPM}(S*G( \sigma_1 ))-\beta_{SMLPM}(\gamma L + \delta M)$

$L=\alpha_{LPM}(L * G(\sigma_1))+\beta_{LPM}(M * G(\sigma_2))$

Where $G(\sigma_{1})$ and $G(\sigma_{2})$ are the normalized Gaussian filters.

Method 2 is also performing convolutions but with non-normalized filters, and we are in charge of deciding whether to normalize or not the filters, this method is useful for becoming crazy and experimenting with weird values. In this method the operations are these:

$D=(L * G_1)+(M * G_2)$

$K=(S * G_3)+( ((L * G_5) + (M * G_6)) * G_4 )$

$L=(L * G_5) + (M * G_6)$

Where $G_1,...,G_6$ are non normalized Gaussian filters.

Setting values with Method 1

The values of the Gaussian functions and the constants $\alpha$, $\beta$, $\gamma$ and $\delta$ can be settled in the Configuration.xml file, in the part of LGN CONFIG

	<!--LGN CONFIG-->
	<LGNMethod>1</LGNMethod>
	<KERNEL_SIZE>3</KERNEL_SIZE>
	<UPPER_KERNEL_SIGMA>0.25f</UPPER_KERNEL_SIGMA>
	<LOWER_KERNEL_SIGMA>1.25f</LOWER_KERNEL_SIGMA>
	<LMM_ALPHA>3.2f</LMM_ALPHA>
	<LMM_BETA>3f</LMM_BETA>
	<SMLPM_ALPHA>1.2f</SMLPM_ALPHA>
	<SMLPM_BETA>1.4f</SMLPM_BETA>
	<SMLPM_GAMMA>0.6f</SMLPM_GAMMA>
	<SMLPM_DELTA>0.4f</SMLPM_DELTA>
	<LPM_ALPHA>0.6f</LPM_ALPHA>
	<LPM_BETA>0.4f</LPM_BETA>

The representation of the Gaussians used for the Simple Opponent process can be seen in the next image, where it shows the upper and lower kernel with their respective $\sigma$.

Setting values with Method 2

The second method consist in edit directly the Kernel filters with the Receptive Field List editor, for that it is needed to change in the configuration file the value of to 2, like this:

<LGNMethod>2</LGNMethod>

Then, we can go to the Receptive Field List tool and choose the folder RFLGN

In the editor, we can modify more parameters. However, the results can be messy because of the freedom of editing the Gaussian filters. It's important to specify the target of each filter; in the image, the target is specified in the columns comb; in this case, the first filter is acting over the L activations and the second on the M activations.

In the case of the S-(L+M) file, the target of the lower filter must be written as LPM.

For editing the filter is better to Open the Gaussian Visualizer, for that we select the upper filter and we can click Copy to visualizer; then we need to activate the checkbox of Gauss 2. In the Receptive Field List, select the row of the lower filter and select the left button copy, and clock paste in the button of the second gaussian.

You can adjust the parameters of the first and second gaussian. For normalizing the two filters, click the button normalize of each filter; the normalization will make that the summation of every pixel of the filter will be 1, and in the case of the difference of gaussian, it will be 0.

If you don to want to normalize the filters, it is also recommended that the summation of the filters will be 0, the effect of increasing the amplitude of the gaussian is an enhancement of the contrast. For that, you can press the button Balance with 0, and the summation of the two filters will be 0.

Visualizing activations from the Simple Opponent Cells

The activations of the Simple Opponent Cells can be seen on the right side of the activations of the LMS cones of the retina, and also a label indicating which activation is showing when you enter with the cursor to the image.

V1 Double Opponent Cells

The name of the java class that performs the process is V1DoubleOpponent.java

The double opponent cells of V1 perform another opponent process similar to the one performed by the LGN.

This obtains the matrices D'K'L' from the matrices DKL coming from the LGN. The contention of these matrices is performed by the following operations:

$D'=\alpha_D ( D * f_{D1})+\beta_D ( D * f_{D2})$

$K'=\alpha_K ( K * f_{K1})+\beta_K ( K * f_{K2})$

$L' = L$

Where $\alpha$ and $\beta$ are constants, and $f_{D1},f_{D2},f_{K1},f_{K2}$ are the convolution filters. For a better understanding, the visualization of the operation can be observed below:

Configuring the parameters and filters

The shape of the filters can be edited in the Receptive Field List by selecting the folder RFDO where there are two files: OpponentD and OpponentK, and each file contains 4 Gaussians, for example, in the OpponentD file, the $f_{D1}$ filter correspond of the composition of two filters which target (Settled in the column comb) is D1.

The second filter $f_{D2}$ is the inverted filter $f_{D1}$; the summation of all the filters should be 0.

For setting the constant values $\alpha$ and $\beta$, we need to modify the values in the Configuration.xml file

	<!--DOUBLE OPPONENT CONFIGURATION-->
	<D_ALPHA>3f</D_ALPHA>
	<D_BETA>3f</D_BETA>
	<K_ALPHA>2f</K_ALPHA>
	<K_BETA>2f</K_BETA>

The activations of the Double Opponent Cells can be seen to the right of the Simple Opponent Cells of the LGN:

V4Color

Color labels assignment

In this stage, there is an assignment of color labels. The activations D', K,' and L' coming from the Double Opponent Cells can be represented in a cylinder where the horizontal axes represent the activations D', K' and the vertical axis represent L' (corresponding to the luminance). The cylinder is divided into parts that correspond to the color labels, as shown in the next figure:

Here a color label matrix is created where each element of that matrix contains 3 labels:

int[] colorLabel = {getConcentricCircleLabel(D, K), getAngleLabel(D, K), getHeightLabel(L)};

getConcentricCircleLabel(D, K) obtain the radius label, which represents the saturation; getAngleLabel(D, K) obtains the angle in the cylinder, representing the hue, and getHeightLabel(L) represents the luminance label.

For visualizing the labels, it is used the HSB color space, where in this code is converted to that space:

             float angle=(float) ((float)getAngleLabel(D, K)/(float)Config.NoRadialDivisions)-Config.HueShift;
             float radial=(float) ((float)getConcentricCircleLabel(D, K)/(float)Config.NoConcentricCircles);
             float h=(float) ((float)getHeightLabel(L)/(float)Config.NoHeightDivisions);
                
             Color color = Color.getHSBColor(angle,radial,h);

Setting the parameters of the color labels

The parameters of the color labels are settled in the file Configuration.xml

	<!--V4 COLOR-->
	<HueShift>0.20f</HueShift>
	<NoConcentricCircles>10</NoConcentricCircles>
	<NoRadialDivisions>10</NoRadialDivisions>
	<NoHeightDivisions>20</NoHeightDivisions>

Where HueShift is the shift in the angle

Visualizing the color labels

The color labels can be viewed below the activations of the Double Opponent Cells: