Jul 29 - Aug 4, 2023
Technology & Thought in Sonic Media (TTSM) - is a series of FREE live online workshops by Ph.D. students and candidates on topics of Music Technology and Sound Studies. Academic knowledge is represented in a short and effective format. This Summer tutors from the Music Department of the University of California San Diego (UCSD) share insides and skills related to their research. Music Generation, Neural Networks and AI, Sound Synthesis, Jazz and Social Research, Conceptual Worldmaking, Epistemology of Electronic Music Instruments, etc.. these can be the tags for this year's TTSM Summer School.
I am the instructor for the first two-day workshop. On the first day, I introduced how to process a typical MIDI file in Python. Tutorial 1 covers the structure and meaning of MIDI metadata and how to extract musical information from it. The processed data could be used in various tasks such as algorithmic composition, statistical corpus analysis, and our second neural network music generation workshop.
Based on the data processing techniques we developed on the first day, on the second day, we used the processed data to train a neural network for music generation. As our input is processed as sequential monophonic pieces, the most suitable neural network model would be an autoregressive model, such as RNN. Tutorial 2 introduced how to build one's own RNN model by writing the formulas and functions from scratch, instead of applying a pre-built library. This exercise gives students a deeper understanding of the mechanism of neural networks, and also demonstrates how to flexibly design and train neural networks rather than being constrained by the settings of other programmers.
Based on the tutorials, I designed two homework accordingly. After the two-day intensive workshop, the students would receive the homework first. By digesting the content and trying to write the codes by themselves, in the mid of the week (after 3 days), the tutorials would be released for students to check their answers. I hope this process could promote learning and motivate the students to make the most out of the workshop.
Somehow unexpectedly, my main audience in this workshop is musicians and creative artists. Then I designed this lighthearted file for people without strong elementary programming skills but still interested in applying the results to some of their projects or just experimenting with new ideas. However, it's worth pointing out that the final generation stage is not the focus of this series. The goal of this workshop is to deliver some fundamental knowledge in MIDI and algorithmic music generation, as well as the basic structures of such systems. The generating result is a side product of this endeavor, and since it's a fundamental teaching series, the generated results are not well-tuned by advanced neural networks or large-scale datasets. We are using a toy model to perform some elementary exercises for teaching purposes. This immature system is uncomparable to the industrial machine learning platforms in the market.
- Super thanks for this incredible workshop and really high NMPM (New Material Per Minute) -- media artist, sound experimenter
- I really wanted to bring concrete results to the workshop on my part (and I'm working on it!), but I'm afraid it will take me more time. I'm currently working on the original second lesson and learning the Python and RNN paradigms in parallel. As soon as the results are (maybe it will take more than a week), I will certainly write to the chat. P.S. I really appreciate your approach (I would say it is fundamental and “Do It Yourself”), because it was exactly what I needed. and I don't think the lack of short-term results is saying anything. P.P.S. I really appreciate participating in the workshop, it is all unique for me. -- experimental sound artist, jazz pianist
- Thank you for the opportunity to get acquainted with many new approaches to working with sound data. This is absolutely new (deep) level for me. And thanks everyone for sharing their bio's and projects -> very inspirational! ps. I am still in the process of doing my homework but looking forward to today's meeting, and to starting using new tools in my research projects as soon as possible. -- sound artist