OpenNAS is an open source VHDL-based Neuromorphic Auditory Sensor (NAS) code generator capable of automatically generating the necessary files to create a VHDL project for FPGA. OpenNAS guides designers with a friendly interface and allows NAS specification using a five-steps wizard for later code generation. It includes several audio input interfaces (AC'97 audio codec, I2S ADC and PDM microphones), different processing architectures (cascade and parallel), and a set of neuromorphic output interfaces (parallel AER, Spinnaker). After NAS generation, designers have everything ready for building and synthesizing the VHDL project for a target FPGA using manufacturer's tools.
OpenNAS is a five-steps wizard which helps you to design and configurate a NAS in an easy way. Each step is focused on a specific section of the sensor, divided in: Commons, Input, Processing and Output. The user can set many parameters, like filters attenuation and number of frequency channels, among others. The meaning of all of these parameters is detailed in the OpenNAS wiki.
This video shows how it works if different architectures.
In this brief tutorial, we will guide the user in how to download, configure and run OpenNAS tool. There is also a videotutorial which summarizes the information showed here on the GitHub repository. You can watch it by clicking on the following image:
OpenNAS has been programmed using Visual Studio Community 2015. Hence, user needs to have installed Visual Studio Community 2015 or greater to compile the project. In addition, Microsoft Windows 7 OS or greater is needed, since the .NET Framework version used in this work is not supported by Microsoft Windows XP OS. For Linux or MAC users, Visual Studio Code is not able to compile the OpenNAS project. Thus, a virtual machine with a Microsoft Windows OS installed have to be used.
OpenNAS has several software dependencies which needs to be solved before executing it in your computer:
- Microsoft .NET Framework 4.5 By clicking in this link you will be redirected to the Microsoft .NET Framework download website. You only need to select the desired language and then click on the "Download" button. Once the download has finished, execute the Microsoft .NET Framework installation file.
- FPGA Synthesis or simulation Suite After HDL generation, users can use FPGA vendors specific tools for synthesis and/or simulation, depending on target hardware paltform.
Files generated by OpenNAS are currently compatible with VHDL versions from VHDL'93 in advance. They are also supported by Verilog 95 and Verilog 2001 versions.
OpenNAS has been tested for synthesis and bitfile generation with the following IDEs:
- Xilinx ISE 14.1.
- Xilinx Vivado (versions from 2016.1 to 2020.1) in several editions: WebPack, Systems Edition, HLS, etc.
- Intel Quartus Prime.
For simulations, OpenNAS has been simulated in:
- ISim from Xilinx ISE 14.1.
- Vivado integrated simulator (versions from 2016.1 to 2020.1).
- ModelSim (versions from PE 9.2B to PE 10.5C) from Intel Quartus Prime.
OpenNAS was conceived as a technology- and manufacturer-independent design in order to allow the community to use it with its preferred FPGA-based platform. According to our possibilities, and with the aim to provide technical support to the users, the OpenNAS output files hab been synthesized and tested in the following FPGA chips:
- Xilinx Spartan 3.
- Xilinx Virtex 5.
- Xilinx Spartan 6.
- Xilinx Artix 7.
- Xilinx Kintex 7.
- Xilinx Zynq-7000 SoC.
- Xilinx Zynq UltraScale+ SoC.
- Intel Cyclone IV.
- Intel Cyclone V.
- Intel Arria 10.
The first step to use OpenNAS software is to download the repository from the RTC-OpenNAS GitHub main webpage. Click on "Clone or download" button, and select "Download ZIP".
When downloaded, extract the project. There is a main folder, OpenNAS, which contains the OpenNAS.sln file and the OpenNAS C# project folder. To run the OpenNAS software tool, open the OpenNAS project by clicking on OpenNAS.sln, and then the VisualStudio environment will be launched. Click on the "Start" button. Then, the Welcome window of the OpenNAS tool will appear.
Once OpenNAS tool has been executed, and the Welcome screen appears, the user only needs to complete the five steps of the OpenNAS wizard to obtain the generated VHDL files.
The Welcome screen shows a brief text which indicates to the user what OpenNAS tool does and also the information about our research group. Click on Next button to move forward in the wizard.
The first step allows to the user to select the NAS common settings, which are the target FPGA chip and its clock frequency, if the NAS is MONO or STEREO, and the number of frequency channels. A picture of the FPGA-based board selected in NAS chip is shown to the user to quickly identify the needed hardware components.
In this step, the input audio source must be selected. There are several options, among which we can find the AC'97 audio codec, a pair of Pulse Density Modulation (PDM) microphones, and an I2S-based audio codec. Each input option has its own configuration parameters, which the user can set according to its project requirements.
NAS processing architecture is defined in step 3, where the user can choose either a cascade or parallel architecture. Besides, filters order and filters output attenuation can be set. Finally, the user can define a frequency range between which the NAS will work, set by default as the human audible sounds range (from 20Hz to 22KHz).
The output interface is selected in the fourth step. As in the most of the event-based neuromorphic devices, the AER protocol is used as output. For this reason, the AER monitor is used as output interface by default and then to connect our neuromorphic sensor to the application jAER and to be able to visualize the NAS output in real-time. However, it is also interesting to connect the NAS output to others neuromorphic hardware, as the SpiNNaker board, in which Spiking Neural Networks (SNN) can be deployed and it can get as input data the output spikes from the NAS.
Finally, when all previous steps have been done, the destination folder in which the VHDL files are going to be generated has to be selected by the user. It needs to create a new folder by hand, and then select it as the destination folder.
A new message will appear if the generation process was done successfully, indicating the destination folder. Click on the Ok button to close the message. At this point, the process of the NAS generation has finished. Navigating to the destination folder, the user can find all the VHDL files needed to synthetize and generate the NAS .bit file and run it using an FPGA. Apart from VHDL files, OpenNAS also generates an XML file summarizing the parameters selection made by the user.
A set of post-processing tools are available to open, analyze, and post-process the output spikes of the NAS. The NAS' output can be logged into a file by using jAER. Then, the spikes file (with .aedat format) can be loaded using one of those tools:
Take a look to those repositories to learn more about neuromorphic audio processing!New functionalities or improvements to the existing project are welcome. To contribute to this project please follow these guidelines:
- Search previous suggestions before making a new one, as yours may be a duplicate.
- Fork the project.
- Create a branch.
- Commit your changes to improve the project.
- Push this branch to your GitHub project.
- Open a Pull Request.
- Discuss, and optionally continue committing.
- Wait untill the project owner merges or closes the Pull Request.
Thank you for contributing in the OpenN@S project!
The author of the OpenN@S' original idea is Angel F. Jimenez-Fernandez, and it is mainly based on the paper "A Binaural Neuromorphic Auditory Sensor for FPGA: A Spike Signal Processing Approach", which was published in 2017 in the journal Transactions on Neural Networks and Learning Systems.
The author would like to thank and give credit to:
- Robotics and Technology of Computers Lab. from the University of Sevilla (Spain).
- Advanced Processor Technologies group from the University of Manchester (UK), for letting us include the SpiNNaker-AER interface in this project.
- Gabriel Jimenez-Moreno, for his support and contribution.
This project is licensed under the GPL License - see the LICENSE.md file for details. Third party software components are under MSPL license
Copyright © 2016 Ángel F. Jiménez-Fernández
ajimenez@atc.us.es
APA: Gutierrez-Galan, D., Dominguez-Morales, J. P., Jimenez-Fernandez, A., Linares-Barranco, A., & Jimenez-Moreno, G. (2021). OpenNAS: Open Source Neuromorphic Auditory Sensor HDL code generator for FPGA implementations. Neurocomputing.
ISO 690: GUTIERREZ-GALAN, D., et al. OpenNAS: Open Source Neuromorphic Auditory Sensor HDL code generator for FPGA implementations. Neurocomputing, 2021.
MLA: Gutierrez-Galan, D., et al. "OpenNAS: Open Source Neuromorphic Auditory Sensor HDL code generator for FPGA implementations." Neurocomputing (2021).
BibTeX: @article{gutierrez2021opennas, title={OpenNAS: Open Source Neuromorphic Auditory Sensor HDL code generator for FPGA implementations}, author={Gutierrez-Galan, D and Dominguez-Morales, JP and Jimenez-Fernandez, A and Linares-Barranco, A and Jimenez-Moreno, G}, journal={Neurocomputing}, year={2021}, publisher={Elsevier} }
APA: Jiménez-Fernández, A., Cerezuela-Escudero, E., Miró-Amarante, L., Domínguez-Morales, M. J., Gomez-Rodriguez, F., Linares-Barranco, A., & Jiménez-Moreno, G. (2017). A Binaural Neuromorphic Auditory Sensor for FPGA: A Spike Signal Processing Approach. IEEE Trans. Neural Netw. Learning Syst., 28(4), 804-818.
ISO 690: JIMÉNEZ-FERNÁNDEZ, Angel, et al. A Binaural Neuromorphic Auditory Sensor for FPGA: A Spike Signal Processing Approach. IEEE Trans. Neural Netw. Learning Syst., 2017, vol. 28, no 4, p. 804-818.
MLA: Jiménez-Fernández, Angel, et al. "A Binaural Neuromorphic Auditory Sensor for FPGA: A Spike Signal Processing Approach." IEEE Trans. Neural Netw. Learning Syst. 28.4 (2017): 804-818.
BibTeX: @article{jimenez2017binaural, title={A Binaural Neuromorphic Auditory Sensor for FPGA: A Spike Signal Processing Approach.}, author={Jim{\'e}nez-Fern{\'a}ndez, Angel and Cerezuela-Escudero, Elena and Mir{\'o}-Amarante, Lourdes and Dom{\'\i}nguez-Morales, Manuel Jesus and Gomez-Rodriguez, Francisco and Linares-Barranco, Alejandro and Jim{\'e}nez-Moreno, Gabriel}, journal={IEEE Trans. Neural Netw. Learning Syst.}, volume={28}, number={4}, pages={804--818}, year={2017}