DESCRIPTION
This repository contains two Jupyter notebooks used in the devopment of a three dimensional binaural sonification prototype that explores the STELIB stellar spectra library from the Spanish Virtual Observatory.
The first notebook (6D-VAE_STELIB-augmented.ipynb), is used to generate a six-dimensional variational autoencoder model that converts the stellar spectra into six-notes musical chords.
The second notebook (STELIB_Multimodal.ipynb), extracts the coordinates of the objects, calculates the graphics, converts the spectra to chords, and sends the output images and OSC messages to the multimodal display implemented in Cabbage/CSound.
The multimodal display is also provided in this repository as a .csd file (STELIB-Explorer.csd).
This research is based on data from the STELIB service developed by the Spanish Virtual Observatory in the framework of the IAU Comission G5 Working Group : Spectral Stellar Libraries.
CONTENTS
- Jupyter notebook 1: 6D-VAE_STELIB-augmented.ipynb
- Jupyter notebook 2: Stelib-Multimodal.ipynb
- CSound/Cabbage file: STELIB-Explorer.csd
- Sample images: Sky.png, Star.png, and Plot.png
- Cabbage mask files: Init.png, Init2.png, and Init3.png
- HRTF files: hrtf-48000-left.dat, and hrtf-48000-right.dat
- LICENSE
- README
- requirements.txt
INSTALLATION
1- Download and install CSound 6.15 from: https://github.com/csound/csound/releases/tag/6.15.0
2- Download Cabbage from (current version 2.9.0): https://cabbageaudio.com/download/
3- Install only Cabbage from the downloaded Cabbage package.
Warning: Current Cabbage version 2.9.0 allows to optionally install the latest version of CSound. This default option should be unchecked not to overwrite CSound 6.15. Latests versions of CSound require additional plugins to work with the image CSound opcodes, so they should not be used.
4- Download the data from: http://svocats.cab.inta-csic.es/stelib/index.php?action=search
5- Download all the content of this repository into the same folder
6- Launch Cabbage, open Stelib_Explorer.csd, and press play
Important note: Cabbage 2.9.0 requires CamelCase update via: File/Convert Identifiers to camelCase. Also change manually “PluginID” to “PulginId” to avoid warnings.
7- Add your path to the downloaded files and run all the cells of Jupyter notebook 2 (Stelib-Multimodal.ipynb) to the sequence
Enjoy the sonifications!
Developer system info: Python 3.8.5 - Cabbage 2.5.0 - i7 macOS 10.15.7 - 32 GB - 1536 MB