3 Specifications
The middleware part of the implementation is composed of the following nodes:
RETINA: It is in charge of the transduction, this retina only includes the cones (A retina with rods is a future project)
- RetinaProcess: It is in charge of transduction, converting RGB images into activations of the LMS cones.
- BasicMotion: Perform motion detection by lighting difference, creating a binary matrix with the areas where there is movement or change.
LGN: The Lateral Geniculate Nucleus is a relay center of information in the Thalamus, it also has cells that perform the simple opponent color processes from the information provided by the retina cones.
-
LGNSimpleOpponentCells: Convert LMS activations to DKL color space. The DKL space is obtained by performing a difference of the activations of the cones where:
$D=L-M$ $K=S-(L+M)$ $L=L+M$
V1: A module representing the V1 Visual Cortex, this cortex is mainly responsible for the selectivity of simple visual features like oriented edges, spatial frequency, etc.
-
V1DoubleOpponent: Performs the double opponent process on the DKL matrices, obtaining the D'K'L' matrices. The double opponent performs a kind of difference of the activations D and K:
$D'=(\alpha_D D * f_1) - (\beta_D D *f_2)$ $K'=(\alpha_k K * f_1) - (\beta_k K *f_2)$ Where the activations D and K are convolved with filters
$f_1$ and$f_2$ , and$\alpha + \beta = 1$ . -
V1SimpleCells: Obtaining edges with specific orientations through convolutions with Gabor filters. It performs the process for obtaining the activations of the cells selective to the stimulus in the middle of the receptive field and in the periphery.
Receptive fields that are selective to the oriented stimuli in the center (first image) and periphery (second)
- V1ComplexCells: Obtaining the spatial invariance of the edges, the spatial invariance means that the receptive field is selective to the oriented stimuli regardless of is in the center or the periphery.
- V1HyperComplex: Obtaining the selectivity of edges with specific lengths, where the activations of the complex cells are convolved by a filter with two inhibitory flanks.
- V1BinocularSimpleCells: Obtains activation of binocular simple cells summing the activations of the monocular simple cells of each eye.
- V1BinocularComplexCells: Obtains activation of binocular complex cells and normalized complex cells. The activation of the binocular complex cells is obtained in the same way as the monocular complex cells.
- V1BinocularMergeProcess: It performs a fusion of the binocular activations of different filter types in order to reduce the binocular noise produced by fake positive signals.
- V1MotionCellsNew: Detect the perpendicular motion of the oriented edges; the method is different compared to 2018 implementation.
V2: A module that represents V2 area, that area has selectivity to angular stimulus and simple curvatures that are a combination of the V1 information.
- V2AngularCells: Obtaining angular selectivity joining the activations of V1 cells, and it also performs the angular invariance.
- V2CurvatureCells: Specific curvature maps of specific radius and direction are obtained.
- V2CornerMotion: Obtaining the motion of corners.
V3: The functions of V3 should be in V2 or V4.
- V3DisparityRange: Performs the relative disparity calculation; this calculation is performed with a weighted sum of the absolute disparities, and a gaussian function is used to set the weights.
V4: V4 area has neurons that respond to more complex visual stimuli.
- V4SimpleShapeCells: Performs simple shape selectivity without scale invariance. The shape selectivity consists of joining certain combinations of activations from the corners and the curvatures.
- V4SimpleShapeScaleInv: Join the simple shape patches of various scales into an activation map.
- V4Color: Assign the color labels. For that, the D'K'L' activations from the double opponent cells are represented in a cylinder where D' and K' are the horizontal axes of the cylinder, and L' is the vertical axis.
MT: MT is important in motion selectivity, in this area the individual signals of motion are integrated to start to calculate the real motion
- MTComponentCells: Merges V1 motion maps from different filter types, plus reduces the resolution to avoid making the process very heavy for the computer.
- MTPatternCells: Calculates the pattern motion using the Intersection of Constrains method with two-component velocity vectors, then it creates a matrix of colors to display the activations.
MST: Performs the motion selectivity of more complex stimuli
- MSTPolarCells: Get the polar motion matrices. For that, a matrix from the LGN is transformed to log-polar coordinates, where the problem is reduced to detecting the motion of oriented edges.
- MSTTemplateCells: Not implemented
Alert: Area for testing a primitive function that has to do with alert
- AlertProcess: A test function that simulates the alert system, where there are three levels of alert, and each level is associated with a frame rate of the system.
TestAttention: Area made for testing the modulation mechanism
- FeedbackProcess: Process to test the modulation mechanism, it sends a modulation matrix to V2 and V2 areas, and the modulation effect can be seen in the system.
This is the activity diagram where the arrows correspond to the spikes sent to small nodes
The system package diagram is as follows.
├───generator
│ │ DirectoryFrame.form
│ │ DirectoryFrame.java
│ │ Java2sAutoComboBox.java
│ │ Java2sAutoTextField.java
│ │ NodeGenerator.form
│ │ NodeGenerator.java
│ │ ProcessList.java
│ │ SmallNodeList.form
│ │ SmallNodeList.java
│ │ test.java
│ │ visualizerTest.form
│ │ visualizerTest.java
│ │
│ └───graph
│ GArea.java
│ GSmallNode.java
│ MGraph.java
│
├───gui
│ │ ActivityFrame.java
│ │ Controls.form
│ │ Controls.java
│ │ ControlsPanel.form
│ │ ControlsPanel.java
│ │ Frame.form
│ │ Frame.java
│ │ FrameActivity.java
│ │ GUI.form
│ │ GUI.java
│ │ GUIActivity.java
│ │ GUITest.form
│ │ GUITest.java
│ │ ImageSelection.java
│ │ Listener.java
│ │ RetinaPanel.java
│ │ RetinaPanel3.form
│ │ RetinaPanel3.java
│ │ ToolsJPanel.form
│ │ ToolsJPanel.java
│ │ Visualizer.java
│ │ VisualizerFrame.form
│ │ VisualizerFrame.java
│ │
│ └───components
│ ListPanel.form
│ ListPanel.java
│ VisPanel.form
│ VisPanel.java
│
├───matrix
│ ArrayMatrix.java
│ DoubleArray.java
│ FloatLabel.java
│ FloatLabelMatrix.java
│ Label.java
│ labelMatrix.java
│ matrix.java
│ MatrixSerialization.java
│ MultiChannelMatrix.java
│ SimpleCellMatrix.java
│
├───middlewareVision
│ ├───config
│ │ AreaNames.java
│ │ Init.java
│ │ XMLReader.java
│ │
│ └───nodes
│ ├───External
│ │ Alert.java
│ │ AlertFrame.form
│ │ AlertFrame.java
│ │ AlertProcess.java
│ │ AttentionTrigger.form
│ │ AttentionTrigger.java
│ │ FeedbackProccess.java
│ │ TestAttention.java
│ │
│ └───Visual
│ ├───LGN
│ │ LGN.java
│ │ LGNSimpleOpponentCells.java
│ │
│ ├───MST
│ │ MST.java
│ │ MSTPolarCells.java
│ │ MSTTemplateCells.java
│ │
│ ├───MT
│ │ MT.java
│ │ MTComponentCells.java
│ │ MTPatternCells.java
│ │
│ ├───Retina
│ │ BasicMotion.java
│ │ Retina.java
│ │ RetinaProccess.java
│ │
│ ├───V1
│ │ ReichardtMotion.java
│ │ V1.java
│ │ V1BinocularComplexCells.java
│ │ V1BinocularMergeProcess.java
│ │ V1BinocularSimpleCells.java
│ │ V1ComplexCells.java
│ │ V1DoubleOpponent.java
│ │ V1HyperComplex.java
│ │ V1MotionCells.java
│ │ V1MotionCells2.java
│ │ V1MotionCellsNew.java
│ │ V1SimpleCells.java
│ │
│ ├───V2
│ │ V2.java
│ │ V2AngularCells.java
│ │ V2CornerMotion.java
│ │ V2CurvatureCells.java
│ │
│ ├───V3
│ │ V3.java
│ │ V3DisparityRange.java
│ │
│ └───V4
│ V4.java
│ V4Color.java
│ V4SimpleShapeCells.java
│ V4SimpleShapeScaleInv.java
│
├───MiniPrograms
│ CurvatureRF.form
│ CurvatureRF.java
│ DisparityList.form
│ DisparityList.java
│ GaborFilterVisualizer.form
│ GaborFilterVisualizer.java
│ GaborList.form
│ GaborList.java
│ GaussianVisualizer.form
│ GaussianVisualizer.java
│ IoCImplementation.form
│ IoCImplementation.java
│ MotionV1Speeds.form
│ MotionV1Speeds.java
│ RF.java
│ RFGeneratorNew.form
│ RFGeneratorNew.java
│ RFlist.java
│
├───spike
│ Location.java
│ LongSpike.java
│ Modalities.java
│
├───utils
│ │ ColorUtils.java
│ │ Config.java
│ │ Convertor.java
│ │ FileUtils.java
│ │ FilterUtils.java
│ │ Functions.java
│ │ GaussianFilter.java
│ │ layoutManager.java
│ │ MathFunctions.java
│ │ MatrixUtils.java
│ │ MotionLabelIndex.java
│ │ Msg.java
│ │ numSync.java
│ │ PairFilter.java
│ │ Scalr.java
│ │ SpecialKernels.java
│ │
│ └───filters
│ CurvatureFilter.java
│ GaborFilter.java
│
└───VisualMemory
│ Cell.java
│ InitCellMemory.java
│ LabeledCells.java
│ MotionCell.java
│
├───LGNCells
│ LGNBank.java
│ SimpleOpponentCells.java
│
├───MSTCells
│ MSTPolar.java
│
├───MTCells
│ ComponentCells.java
│ MTBank.java
│ PatternCells.java
│
├───V1Cells
│ ComplexCells.java
│ DoubleOpponentCells.java
│ HypercomplexCells.java
│ MotionCellsV1.java
│ SimpleCells.java
│ StereoscopicCells.java
│ StereoscopicMergedCells.java
│ V1Bank.java
│
├───V2Cells
│ AngleCells.java
│ CornerMotionCells.java
│ CurvatureCells.java
│ V2Bank.java
│
├───V3Cells
│ StereoscopicRangeCells.java
│ V3Bank.java
│
└───V4Cells
ScaleMergeCells.java
SimpleShapeCells.java
V4Bank.java