diff --git a/NET.sln b/NET.sln index eaca6a1..3b8419c 100644 --- a/NET.sln +++ b/NET.sln @@ -1,14 +1,14 @@  Microsoft Visual Studio Solution File, Format Version 12.00 -# Visual Studio Express 14 for Windows Desktop -VisualStudioVersion = 14.0.24720.0 +# Visual Studio 15 +VisualStudioVersion = 15.0.27703.2000 MinimumVisualStudioVersion = 10.0.40219.1 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DemoConsoleApp", "Demo\DemoConsoleApp\DemoConsoleApp.csproj", "{D18DB05D-F4C4-4388-846A-52BA4B3722D1}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{27FD896B-0D8F-4083-A577-2CA808E4145E}" ProjectSection(SolutionItems) = preProject LICENSE.txt = LICENSE.txt - Readme.txt = Readme.txt + Readme.md = Readme.md EndProjectSection EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RCNet", "RCNet\RCNet.csproj", "{7F44508E-0B06-4652-9631-4740EA97EFE1}" @@ -31,4 +31,7 @@ Global GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE EndGlobalSection + GlobalSection(ExtensibilityGlobals) = postSolution + SolutionGuid = {FE2CC0EB-F816-4781-8AA1-D373BBAB50FA} + EndGlobalSection EndGlobal diff --git a/Readme.md b/Readme.md new file mode 100644 index 0000000..6d8edb1 --- /dev/null +++ b/Readme.md @@ -0,0 +1,89 @@ +# Reservoir Computing for .NET (RCNet) +The aim of the project is to make the [reservoir computing](https://en.wikipedia.org/wiki/Reservoir_computing) methods easy to use and available for .net platform without dependency on external heterogeneous libraries. +Two main reservoir computing methods are called Echo State Network (ESN) and Liquid State Machine (LSM). +The implemented solution supports both of these methods. However, since ESN and LSM are based on very similar principles, RCNet brings the option to combine them at the same time. This approach, as I believe, opens up new interesting possibilities. This general implementation is called "**State Machine**" in the context of RCNet. +Contact: +oldrich.kozelsky@email.cz + +## Technical information + - Source code is written in C# 6.0 + - Most components are serializable + - Backward compatibility is not guaranteed + - RCNet documentation is located on [wiki pages](https://github.com/okozelsk/NET/wiki) + +## Demo application +Main RCNet functionality is demonstrated in a simple demo application (/Demo/DemoConsoleApp). Application has no startup parameters, all necessary settins are specified in DemoSettings.xml file. DemoSettings.xml has to be in the same folder as the executable DemoConsoleApp.exe. Application performs sequence of demo cases defined in DemoSettings.xml. The input data for each demo case must be stored in the csv file. You can modify DemoSettings.xml and configure your own tasks or modify and tune existing ones. + + +## Main implemented components +### Data handling +|Component|Description| +|--|--| +|BasicStat|Implements the simple and thread safe statistics| +|Normalizer|Data normalization/denormalization, Gaussian standardization| +|PatternBundle|Bundle of pattern and desired output vector| +|PatternDataLoader|The class allows to upload sample data for a Classification or Hybrid task from a csv file| +|TimeSeriesBundle|Bundle of input vector and desired output vector| +|TimeSeriesDataLoader|The class allows to upload sample data for a Prediction task from a csv file| +|BundleNormalizer|Helper class for easy standardization and normalization/naturalization of sample data bundle| + +### Math +|Component|Description| +|--|--| +|ODENumSolver|Ordinary Differential Equations (ODE) Numerical Solver (Euler and RK4 methods)| +|Vector|Class represents the mathematical vector of double values| +|Matrix|Class represents the mathematical matrix of double values| +|EVD|Eigenvalues and eigenvectors of a real matrix| +|QRD|QR Decomposition| +|HurstExpEstim|Class implements calculation of the Hurst exponent estimation (rescalled range method). Class is not used by RCNet.| +|PhysUnit|Encaptualates SI physical unit| + +### Analog neuron activation functions +|Component|Description| +|--|--| +|BentIdentity|Bent identity activation function| +|Elliot|Elliot activation function (aka Softsign)| +|Gaussian|Gaussian activation function| +|Identity|Identity activation function (aka Linear)| +|ISRU|ISRU (Inverse Square Root Unit) activation function| +|LeakyReLU|Leaky ReLU (Leaky Rectified Linear Unit) activation function| +|Sigmoid|Sigmoid activation function| +|Sinc|Sinc activation function| +|Sinusoid|Sinusoid activation function| +|SoftExponential|Soft exponential activation function| +|SoftPlus|Soft Plus activation function| +|TanH|TanH activation function| +See the [wiki pages.](https://en.wikipedia.org/wiki/Activation_function) + +### Spiking neuron activation functions +|Component|Description| +|--|--| +|SimpleIF|Simple Integrate and Fire activation function| +|AdSimpleIF|Adaptive Simple Integrate and Fire activation function| +|LeakyIF|Leaky Integrate and Fire activation function| +|ExpIF|Exponential Integrate and Fire activation function| +|AdExpIF|Adaptive Exponential Integrate and Fire activation function| +See the [wiki pages.](https://en.wikipedia.org/wiki/Biological_neuron_model) + +### Non-recurrent Networks +|Component|Description| +|--|--| +|FeedForwardNetwork|Implements the feed forward network supporting multiple hidden layers. (Xml constructor)| +|LinRegrTrainer|Implements the linear regression trainer of the feed forward network. This is the special case trainer for FF network having no hidden layers and Identity output activation function. (Xml constructor)| +|RPropTrainer|Resilient propagation trainer (iRPROP+ variant) of the feed forward network. (Xml constructor)| +|ParallelPerceptron|Implements the parallel perceptron network. (Xml constructor)| +|PDeltaRuleTrainer|P-Delta rule trainer of the parallel perceptron network. (Xml constructor)| + +### State Machine Components +|Component|Description| +|--|--| +|StaticSynapse|Static synapse computes constantly weighted signal from source to target neuron| +|InputAnalogNeuron|Input neuron is the special type of very simple neuron. Its purpose is only to mediate input analog value for a synapse| +|InputSpikingNeuron|Spiking input neuron is the special type of neuron. Its purpose is to preprocess input analog value to be deliverable as the spike train signal into the reservoir neurons through a synapse| +|ReservoirAnalogNeuron|Reservoir neuron is the main type of the neuron processing input stimuli and emitting output signal. Analog neuron produces analog output. Main features: Retainment (leaky integrator), The second power as augmented readout state| +|ReservoirSpikingNeuron|Reservoir neuron is the main type of the neuron processing input stimuli and emitting output signal. Spiking neuron produces spikes. Main features: Firing rate as primary readout state, Membrane potential as augmented readout state| +|Reservoir|Implements reservoir supporting analog and spiking neurons working together. (Xml constructor). Main features: SpectralRadius, Multiple 3D pools of neurons, Pool to pool connections. It can work as the Echo State Network reservoir, Liquid State Machine reservoir or Mixed reservoir| +|ReadoutUnit|Contains the trained unit associated with output field and related important error statistics. Trained unit can be the Feed Forward Network or the Parallel Perceptron Network| +|ReadoutLayer|Class implements the common readout layer for the reservoir computing methods. (Xml constructor). Supports x-fold cross validation method.| +|StateMachine|Implements the State Machine Network. (Xml constructor). Supports multiple internal reservoirs. Task types: Prediction, Classification, Hybrid| + diff --git a/Readme.txt b/Readme.txt deleted file mode 100644 index b90aaf4..0000000 --- a/Readme.txt +++ /dev/null @@ -1,129 +0,0 @@ -The aim of this project is to make the reservoir computing methods easy to use and available for .net without dependency on external libraries. - -Currently implemented components --------------------------------- -Normalizer - Data normalization/denormalization - Gaussian standardization - -Ordinary Differential Equations (ODE) Numerical Solver - Euler - RK4 - -Activation functions - Analog - BentIdentity - Elliot - Gaussian - Identity - ISRU - LeakyReLU - Sigmoid - Sinc - Sinusoid - SoftExponential - SoftPlus - TanH - Spiking - SimpleIF (Simple Integrate and Fire) - LeakyIF (Leaky Integrate and Fire) - ExpIF (Exponential Integrate and Fire) - AdSimpleIF (Adaptive Simple Integrate and Fire) - AdExpIF (Adaptive Exponential Integrate and Fire) - -Random distributions - Uniform - Gaussian - -Feed Forward Network - Xml constructor - Multiple hidden layers - Trainers - Resilient propagation trainer (iRPROP+ variant) - Linear regression trainer (white noise stabilization) - -Parallel Perceptron Network - Xml constructor - Trainers - P-Delta Rule Trainer - -Reservoir Neuron - Input - Analog - Spiking - Converts analog input to spike train - Hidden - Augmented readout state feature - Analog - Retainment (leaky integrator) feature - The second power as augmented readout state - Spiking - Firing rate as primary readout state - Membrane potential as augmented readout state - -Reservoir - Xml constructor - Provides important internal statistics - Supports SpectralRadius parameter - Multiple 3D pools of neurons - Pool internal connections - Supports Euklidean Distance property - Pool to pool connections - Supports mixing of analog and spiking neuron pools - so it can work as - Echo State Network reservoir - Liquid State Machine reservoir - Hybrid (mixed) reservoir - -State Machine - Xml constructor - Supports multiple internal reservoirs - Supported task types: - Prediction - Time series input - Output is value prediction - Classification - Pattern input - Supports variable length of patterns - Output is probability of the class - Hybrid - Pattern input - Supports variable length of patterns - Output is value prediction - -Readout unit - FF Network or P-Perceptron - -Readout layer - Xml constructor - Independent on predictors generator (State Machine) - Supports x-fold cross validation method - Cluster of readout units per each output field - -Bundle normalizer - Helper for data bundle normalization - -Bundle Data loaders (csv) - PatternDataLoader (Classification or Hybrid task) - TimeSeriesDataLoader (Prediction task) - -Miscellaneous - Queue (thread safe) - Hurst exponent estimator (the toughest variant) - (others) - -Demo application ----------------- -Main functionality is demonstrated in a simple demo application (/Demo/DemoConsoleApp). Application has no startup parameters, all necessary settins are specified in DemoSettings.xml file. DemoSettings.xml has to be in the same folder as the executable DemoConsoleApp.exe. Application performs sequence of demo cases defined in DemoSettings.xml. Input data has to be stored in a csv file. You can simply modify DemoSettings.xml and configure your own cases to be prformed. - - -Other information ------------------ -Source code is written in C# 6.0. -Most components are serializable. -Backward compatibility is not guaranteed. -Product documentation will be published on the associated wiki. - - -Contact: -oldrich.kozelsky@email.cz