6 Shape processing tutorial
The simple cells of V1 are selective to the borders with specific orientations
To achieve this behavior in computation, convolutions are performed with Gabor filters for each orientation. It's relevant to mention that we obtain 2 types of simple cells here, those which have a receptive field with excitatory zones in the middle and those which have excitatory zones in the periphery.
For
To set the Gabor filters, it's needed to go to Tools > Gabor Bank List and we can create a list of different types of Gabor filters; To visualize the filter, we can press the button Edit filter, and this window will appear.
The parameters to control in the editor are the following:
-
Kernel Size : Is the dimensions of the filter; the dimensions are in a 1:1 proportion.
-
$\sigma$ : Represents the length of the filter -
$\lambda$ : Is the wavelength of the filter -
$\gamma$ : Controls the length and height proportion or the ellipticity -
$\phi$ : Is the phase of the filter; this value will be used for assigning the filters with$\phi_2$ , because all filters with phase$\phi_1$ are settled to 0. -
$\theta$ : Is the angle of the filter; note: The value must be 0 because the filter will be rotated automatically in the detection of borders. -
Kernel angle : Is the rotation of the kernel It also must be 0.
Is important to check that the filter sum should be 0. Once the filter is settled, click on Copy values and paste it into a row of the Gabor Bank List. And finally is important to save the list.
Using different Gabor filters with different frequencies helps to detect orientations with different levels of detail. Filter with higher frequencies detects borders with higher details, and borders with lower frequencies detect coarse details, as shown in the next figure, where the activation maps are divided by orientations.
The portion of the code in charge of making the convolutions is located in the class V1SimpleCells.java where the filters are loaded into the Simple cells. This call the convolution for each orientation, where x1 is the index of the Gabor filter type, x2 is the index of the eye, src is the source image:
void convolve(int x1, int x2, Mat src) {
for (int i = 0; i < Config.gaborOrientations; i++) {
V1Bank.SC[x1][x2].Even[i].mat = filterV1(src, V1Bank.SC[x1][x2].evenFilter[i], 0.2);
V1Bank.SC[x1][x2].Odd[i].mat = filterV1(src, V1Bank.SC[x1][x2].oddFilter[i], 0.2);
}
}and the function filterV1 is the following. It performs a convolution between img and filter, then the values are divided by max for keeping the activations under than 1 and finally a threshold is applied where thresh is the threshold value.
Mat filterV1(Mat img, Mat filter, double thresh) {
//initialize the filter Matrix
Mat filt = Mat.zeros(img.rows(), img.cols(), CvType.CV_32FC1);
img.convertTo(img.clone(), CV_32F);
//perform the convolution
Imgproc.filter2D(img, filt, CV_32F, filter);
//Block for keeping the activations under than 1
double max = Core.minMaxLoc(filt).maxVal;
if (max > 1) {
Core.divide(filt, Scalar.all(max), filt);
}
//Aplying a threshold, where thresh<=x<=1
Imgproc.threshold(filt, filt, thresh, 1, Imgproc.THRESH_TOZERO);
return filt;
}Finally, the activations of the simple cells are located at the right of the V1 Double Opponent activations.
There is nothing to configure at this stage, but anyways, there is an explanation of how the process works:
The receptive fields of the complex cells of V1 are invariant to the position of edges, this means they will have a similar activation if an oriented bar is in the middle or not of the receptive field, as shown in the next figure:
To achieve this, the activations of each oriented simple cell of different phases
The activation of the complex cells are
Where
The portion of code that performs this process is located in V1ComplexCells.java
public void energyProcess(int x1, int x2) {
int x = CC[x1][x2].Cells.length;
for (int i = 0; i < x; i++) {
CC[x1][x2].Cells[i].mat = Functions.sumPowProcess(CC[x1][x2].simpleCells.Even[i].mat,
CC[x1][x2].simpleCells.Odd[i].mat, 2);
}
}Where sumPowProcess function squares the inputs, and then it performs the sum.
The effect of the complex cell activations is smoothing in the border zones.
Finally, the visualization of the Complex Cell activations is located at the right of the Simple Cells:
V1 Hypercomplex cells perform the selectivity to the border with a specific length
For this, an end-stopped filter is used; this filter consists of two inhibitory regions and, in the middle, an excitatory region.
The excitatory region is narrow because the convolution with that region keeps the activation of the previous stage almost the same.
The process is performed with a convolution of the activations from the complex cells with the end-stopped filters, for that various end-stopped filters can be used for having selectivity to different lengths; at the end, the activations are merged to go to the next stage.
The filters of hypercomplex cells can be edited in the Receptive Field List program; for that, we need to choose the folder RFV1HC, where every filter corresponds to a file. The file endstopped is an example that contains two inhibitory regions; for setting a filter with selectivity to borders of 20 pixels, we need to set the values of
Formally for having the selectivity for borders with
We can generate more files with different names, however, is recommended to keep the files endstopped and base.
The base file contains only the excitatory region; in that file, we can set the value of A, which indicates that the proportion of the activations of the complex cells will remain. In the case of the base filter, the recommended values of
The system can show different activations of the hypercomplex cells representing selectivity for different lengths:
However, the setting of this filter can allow a lot of freedom, so it is important to keep the example files in case we want to recover the original values.
The Hypercomplex cells images are located at the right of the Complex Cells activations; the summation of hypercomplex cells is the merge of every hypercomplex cell bank.
The selectivity of corners with specific angles is performed by joining the orientation maps from V1:
Two images from V1 are convolved with filters indicating the direction of the border (Because each border has two directions):
and then the result is passed through an activation function. The activation function is defined as:
Where
The part of the code in V2AngularCells that call the angular activation function is:
for (int j = 0; j < Config.gaborOrientations * 2; j++) {
V2Bank.AC[x1][x2].Cells[i][j].mat = angularActivation(filtered[j], filtered[(i + j + 1) % (Config.gaborOrientations * 2)], l3);Where the activation function in the code is:
public Mat angularActivation(Mat M1, Mat M2, double l) {
Mat dst = new Mat();
Mat vlvr = new Mat();
Mat vlpvr = new Mat();
Mat num = new Mat();
Mat den = new Mat();
Mat h = new Mat();
Scalar dl3 = new Scalar((double) 1 / l);
Scalar d2l3 = new Scalar((double) 2 / l);
Scalar dl3_2 = new Scalar((double) 1 / (l * l));
Core.multiply(M1, M2, vlvr);
Core.add(M1, M2, vlpvr);
Core.add(vlpvr, d2l3, num);
Core.multiply(vlpvr, dl3, den);
Core.add(den, vlpvr, den);
Core.add(den, dl3_2, den);
Core.divide(num, den, h);
Core.multiply(vlvr, h, dst);
Imgproc.threshold(dst, dst, 1, 0, Imgproc.THRESH_TRUNC);
return dst;
}The template for setting the convolution for performing the process can be edited in the Receptive Field List by choosing the folder RFV2.
The template is composed of two gaussian filters; the gaussian filter with a negative value of A is for making the activations sharp. Is important to keep the angle of the filter unmodified because the system will rotate the filter automatically, and also, the filter should be in the north. The template for the angular filter is shown in the next figure:
The results will depend on the configuration of the filters; if the filters have a small value of
When the second filter has some negative value of A, the activations are less intense than when there is no value of A in the surrounding filter.
In the next image, there is another set of filters, the first experiment shows a narrow activation produced with another ON-OFF center-surround filter, and the second is only a gaussian with a wide value of
In the case of setting the system for 4 orientations, the activations are shown by groups of activations of 45,90,135 and 180 degrees; for that, all the matrices with a certain aperture are merged into one matrix, this code in V2AngularCells.java performs the merge process
public void mergeCells(int x1, int x2) {
for (int i = 0; i < V2Bank.AC[x1][x2].mergedAC.length; i++) {
V2Bank.AC[x1][x2].mergedAC[i] = MatrixUtils.maxSum(V2Bank.AC[x1][x2].Cells[i]);
}
}For showing only the activations of all matrices of certain aperture but different directions, we need to modify the code for visualizing these matrices; that code is in the method visualize() , where we need to uncomment the commented code and comment the original code:
public void visualize() {
for (int x0 = 0; x0 < Config.gaborBanks; x0++) {
for (int i = 0; i < V2Bank.AC[0][0].mergedAC.length; i++) {
Visualizer.setImage(V2Bank.AC[x0][0].mergedAC[i], "Angular Cells L", Visualizer.getRow("HC") + 1, i, "AC");
Visualizer.setImage(V2Bank.AC[x0][1].mergedAC[i], "Angular Cells R", Visualizer.getRow("HC") + 2, i, "AC");
}
}
//Uncomment for visualizing the activations of only one aperture and different directions
/*
int angularIndex=3;
for (int i = 0; i < V2Bank.AC[0][0].Cells[0].length; i++) {
Visualizer.setImage(V2Bank.AC[0][0].Cells[angularIndex][i].mat, "Angular Cells L", Visualizer.getRow("HC")+1, i, "AC");
Visualizer.setImage(V2Bank.AC[0][1].Cells[angularIndex][i].mat, "Angular Cells R", Visualizer.getRow("HC")+1, i, "AC");
}*/
}Finally, the place of the V2 Angular activations is right to the V1 Hypercomplex cells.
To perform the selectivity to specific curvatures, the model consists of several Gabor filters located in a radial path as shown in the image:
Where the activation is proportional to the multiplication of a matrix from V1 convolved with each Gabor filter:
For generating the curvature filters, we need to use the Curvature editor, but before that, we need to configure some of the <curvatureImageHeight> and <curvatureImageWidth> values in Configuration.xml and set as the same as <dimensions>; this is to have a good reference for setting the curvature radius and using it in the system:
<curvatureEditorImage>ConfigFiles/circles.JPG</curvatureEditorImage>
<curvatureImageWidth>200</curvatureImageWidth>
<curvatureImageHeight>200</curvatureImageHeight>
The parameters were explained in the section of Configuration of the wiki. However, it is shown how some parameters affect the result.
In the editor, we can choose a file or create one; when the filter is selected with the button generate, we can see the result of the process. The number of filters affects the precision of the result; with more number of filters, the precision is more (This means less tolerance to errors), but the process is slower:
The value
The activation level or brightness can be controlled with the parameter mul_factor. And finally, we can save the filter, is recommended to include in the name the radius of the filter.
However, at the time of visualizing in the program, we need to make adjustments in the editor because it takes the combined activations of V1, so depending on the stimuli, the main adjustments are related to the Gabor filters and the multiplication factor. That process takes some time, so in the future, this module needs to be improved.
The activations are located to the right of the V2 angular activations.
It's important to mention that the curvature activation visualization is merged; this is to say, all the curvatures with different directions are joined into one matrix with all curvatures.
In this process, we obtain the activations to a simple shape pattern of only one scale; for that, the previous activations (Corners and curvatures) are labeled as:
-
For angular activations :
a(n1)(n2)-(i1)(i2)where$n1$ is the index of type of gabor filter,$n2$ is the index of eye.$i1$ is the index of aperture, and$i2$ is the index of direction; for example, the names could be like:a00-10,a00-23,a01-22. -
For curvature activations: Similar to the previous point, but the format is
c(n1)(n2)
For editing the receptive fields of V4, we need to open the tool Receptive Field List, and we need to make sure that the folder RFV4 is selected. Then we can set the gaussian filters that will be part of the simple shape receptive field; the next image shows the receptive field for detecting a simple triangle:
The indexes a0-13, a0-01, and a0-06 were chosen using this image as a reference for four Gabor orientations, where the vertical numbers correspond to the index i1 and the horizontal to i2 for the angular activations:
To clarify more, the next image is shown the V2 angular patch associated with the combination:
When a simple shape patch is designed, we can save the file with different scales to have different scale activations. To achieve that, it is necessary to write in the field Scales the values of the scales separated by a comma, like this:
Then the files will be saved with this name, where after the _ is written the scale number, this will be necessary for joining all the activations with the same name but different scale numbers:
The result can be seen in the next image, which shows the simple scale patches and the multi-scale patches:
