3 Specifications
The middleware part of the implementation is composed of the following nodes:
RETINA: It is in charge of the transduction, this retina only includes the cones (A retina with rods is a future project)
- RetinaProcess: It is in charge of transduction, converting RGB images into activations of the LMS cones.
- BasicMotion: Perform motion detection by lighting difference, creating a binary matrix with the areas where there is movement or change.
LGN: The Lateral Geniculate Nucleus is a relay center of information in the Thalamus, it also has cells that perform the simple opponent color processes from the information provided by the retina cones.
-
LGNSimpleOpponentCells: Convert LMS activations to DKL color space. The DKL space is obtained by performing a difference of the activations of the cones where:
The DKL color space can be represented in a 3D space, as shown in the image
V1:
-
V1DoubleOpponent: Performs the double opponent process on the DKL matrices, obtaining the D'K'L' matrices. The double opponent performs a kind of difference of the activations D and K:
Where the activations D and K are convolved with filters
and 
- V1SimpleCells: Obtaining edges with specific orientations through convolutions with Gabor filters
- V1ComplexCells: Obtaining the spatial invariance of the edges
- V1HyperComplex: Obtaining the selectivity of edges with specific lengths
- V1BinocularSimpleCells: Obtains activation of binocular simple cells
- V1BinocularComplexCells: Obtains activation of binocular complex cells and normalized complex cells
- V1BinocularMergeProcess: It performs a fusion of the binocular activations of different filter types.
- V1MotionCellsNew: Detect edge movement with specific orientations, method is different compared to 2018 implementation
V2:
- V2AngularCells: Obtaining angular selectivity
- V2CurvatureCells: Specific curvature maps are obtained
- V2CornerMotion: Obtaining the movement of corners
V3:
- V3DisparityRange: Performs the relative disparity calculation
V4:
- V4SimpleShapeCells: Performs simple shape selectivity, without scale invariance
- V4SimpleShapeScaleInv: Join the simple shape patches of various scales into an activation map
- V4Color: Assign the color labels
MT:
- MTComponentCells: Merges V1 motion maps from different filter types, plus reduces resolution
- MTPatternCells: Calculates the pattern motion, then it creates an matrix of colors to display
MST:
- MSTPolarCells: Get the polar motion matrices
- MSTTemplateCells: Not implemented
Alert:
- AlertProcess: Change the sample rate or system animation with 3 alert levels
TestAttention:
- FeedbackProcess: Process to test the modulation mechanism, send a modulation matrix to several matrices
This is the activity diagram where the arrows correspond to the spikes sent to small nodes
The system package diagram is as follows
├───generator
│ │ DirectoryFrame.form
│ │ DirectoryFrame.java
│ │ Java2sAutoComboBox.java
│ │ Java2sAutoTextField.java
│ │ NodeGenerator.form
│ │ NodeGenerator.java
│ │ ProcessList.java
│ │ SmallNodeList.form
│ │ SmallNodeList.java
│ │ test.java
│ │ visualizerTest.form
│ │ visualizerTest.java
│ │
│ └───graph
│ GArea.java
│ GSmallNode.java
│ MGraph.java
│
├───gui
│ │ ActivityFrame.java
│ │ Controls.form
│ │ Controls.java
│ │ ControlsPanel.form
│ │ ControlsPanel.java
│ │ Frame.form
│ │ Frame.java
│ │ FrameActivity.java
│ │ GUI.form
│ │ GUI.java
│ │ GUIActivity.java
│ │ GUITest.form
│ │ GUITest.java
│ │ ImageSelection.java
│ │ Listener.java
│ │ RetinaPanel.java
│ │ RetinaPanel3.form
│ │ RetinaPanel3.java
│ │ ToolsJPanel.form
│ │ ToolsJPanel.java
│ │ Visualizer.java
│ │ VisualizerFrame.form
│ │ VisualizerFrame.java
│ │
│ └───components
│ DisparityPanel.form
│ DisparityPanel.java
│ ListPanel.form
│ ListPanel.java
│ VisPanel.form
│ VisPanel.java
│
├───imgio
│ RetinalImageIO.java
│ RetinalTextIO.java
│
├───mapOpener
│ ActivationMaps.java
│ AmapViewer.form
│ AmapViewer.java
│ Convertor.java
│ OpenObjectTest.java
│
├───matrix
│ ArrayMatrix.java
│ DoubleArray.java
│ FloatLabel.java
│ FloatLabelMatrix.java
│ Label.java
│ labelMatrix.java
│ matrix.java
│ MatrixSerialization.java
│ MultiChannelMatrix.java
│ SimpleCellMatrix.java
│
├───middlewareVision
│ ├───config
│ │ AreaNames.java
│ │ Init.java
│ │ XMLReader.java
│ │
│ └───nodes
│ ├───External
│ │ Alert.java
│ │ AlertFrame.form
│ │ AlertFrame.java
│ │ AlertProcess.java
│ │ AttentionTrigger.form
│ │ AttentionTrigger.java
│ │ FeedbackProccess.java
│ │ TestAttention.java
│ │
│ └───Visual
│ ├───LGN
│ │ LGN.java
│ │ LGNSimpleOpponentCells.java
│ │
│ ├───MST
│ │ MST.java
│ │ MSTPolarCells.java
│ │ MSTTemplateCells.java
│ │
│ ├───MT
│ │ MT.java
│ │ MTComponentCells.java
│ │ MTPatternCells.java
│ │
│ ├───Retina
│ │ BasicMotion.java
│ │ Retina.java
│ │ RetinaProccess.java
│ │
│ ├───V1
│ │ ReichardtMotion.java
│ │ V1.java
│ │ V1BinocularComplexCells.java
│ │ V1BinocularMergeProcess.java
│ │ V1BinocularSimpleCells.java
│ │ V1ComplexCells.java
│ │ V1DoubleOpponent.java
│ │ V1HyperComplex.java
│ │ V1MotionCells.java
│ │ V1MotionCells2.java
│ │ V1MotionCellsNew.java
│ │ V1SimpleCells.java
│ │
│ ├───V2
│ │ V2.java
│ │ V2AngularCells.java
│ │ V2CornerMotion.java
│ │ V2CurvatureCells.java
│ │
│ ├───V3
│ │ V3.java
│ │ V3DisparityRange.java
│ │
│ └───V4
│ V4.java
│ V4Color.java
│ V4SimpleShapeCells.java
│ V4SimpleShapeScaleInv.java
│
├───MiniPrograms
│ CurvatureRF.form
│ CurvatureRF.java
│ DisparityList.form
│ DisparityList.java
│ GaborFilterVisualizer.form
│ GaborFilterVisualizer.java
│ GaborList.form
│ GaborList.java
│ GaussianVisualizer.form
│ GaussianVisualizer.java
│ IoCImplementation.form
│ IoCImplementation.java
│ MotionV1Speeds.form
│ MotionV1Speeds.java
│ RF.java
│ RFGeneratorNew.form
│ RFGeneratorNew.java
│ RFlist.java
│
├───spike
│ Location.java
│ LongSpike.java
│ Modalities.java
│
├───utils
│ │ ColorUtils.java
│ │ Config.java
│ │ Convertor.java
│ │ FileUtils.java
│ │ Functions.java
│ │ layoutManager.java
│ │ MathFunctions.java
│ │ MatrixUtils.java
│ │ MotionLabelIndex.java
│ │ Msg.java
│ │ numSync.java
│ │ PairFilter.java
│ │ Scalr.java
│ │ SpecialKernels.java
│ │
│ └───filters
│ CurvatureFilter.java
│ GaborFilter.java
│
└───VisualMemory
│ Cell.java
│ InitCellMemory.java
│ LabeledCells.java
│ MotionCell.java
│
├───LGNCells
│ LGNBank.java
│ SimpleOpponentCells.java
│
├───MSTCells
│ MSTPolar.java
│
├───MTCells
│ ComponentCells.java
│ MTBank.java
│ PatternCells.java
│
├───V1Cells
│ ComplexCells.java
│ DoubleOpponentCells.java
│ HypercomplexCells.java
│ MotionCellsV1.java
│ SimpleCells.java
│ StereoscopicCells.java
│ StereoscopicMergedCells.java
│ V1Bank.java
│
├───V2Cells
│ AngleCells.java
│ CornerMotionCells.java
│ CurvatureCells.java
│ V2Bank.java
│
├───V3Cells
│ StereoscopicRangeCells.java
│ V3Bank.java
│
└───V4Cells
GaussianFilter.java
ScaleMergeCells.java
SimpleShapeCells.java
V4Bank.java