Skip to content

Commit

Permalink
Update paper.md
Browse files Browse the repository at this point in the history
  • Loading branch information
carterenyi authored Apr 7, 2019
1 parent 99764cd commit 7ca1b35
Showing 1 changed file with 8 additions and 6 deletions.
14 changes: 8 additions & 6 deletions paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,32 +9,34 @@ tags:
authors:
- name: Aaron Carter-Ényì
orcid: 0000-0003-2614-4777
affiliation: "1, 2, 3" # (Multiple affiliations must be quoted)
affiliation: "1, 2, 3, 4" # (Multiple affiliations must be quoted)
affiliations:
- name: Assistant Professor, Morehouse College
index: 1
- name: Lecturer, Spelman College
- name: Instructor, Spelman College
index: 2
- name: Fellow, American Council of Learned Societies
- name: Fellow, Fulbright Scholar Program
index: 3
- name: Project Director, Algorithmic Thinking, Analysis and Visualization in Music (ATAVizM)
index: 3
date: 16 October 2018
bibliography: paper.bib
---

# Summary

EMViz (Early Music Visualization) provides built-in pattern recognition for symbolic music (MIDI) based on a contour recursion algorithm by Carter-Enyi [@Carter-Enyi:2016a] producing visualizations of musical form using arc diagrams, as proposed by Wattenberg ([@Wattenberg:2002]). Currently implemented in MATLAB and deployed as a standalone executable (using MATLAB Runtime), EMViz is now available at emviz.org and https://github.com/carterenyi/emviz (MIT license).
EMViz (Early Music Visualization) provides built-in pattern recognition for symbolic music (MIDI) based on a contour recursion algorithm by Carter-Enyi [@Carter-Enyi:2016a] producing visualizations of musical form using arc diagrams, as proposed by Wattenberg [@Wattenberg:2002]. Currently implemented in MATLAB and deployed as a standalone executable (using MATLAB Runtime), EMViz is now available at emviz.org and https://github.com/carterenyi/emviz (MIT license).

A primary learning objective in music theory courses is the ability to analyze musical form, often using a bracketed diagram. This humanistic process often involves listening with a score, stopping the recording for a close reading of the score, flipping back and forth through the document to make comparisons, and so on. While manual analysis is important, the conventional method of summarizing one’s analysis as a bracket diagram is reductive without being illustrative. A promising digital method for visualization of musical form is found in Martin Wattenberg’s Shape of Song, a defunct web app from 2002. The arc diagram visualization technique for Shape of Song is brilliant, but ultimately the project did not live up to its potential because of a poor understanding of how composers develop musical themes. Wattenberg describes his string-matching method in his 2002 paper: “Each arc connects two matching passages, where a ‘match’ means that they contain the same sequence of pitches” [@Wattenberg:2002]. However, musical themes and motives are often transposed or otherwise modified in the course of a piece. A concept learned by music majors is the subject and answer of a fugue (an imitative polyphonic composition). The subject is the initial statement of a fugue’s theme, stated in a monophonic texture (by itself) and the answer is the first interpolation stated in a polyphonic texture (against counterpoint). Sometimes the answer is an exact transposition (real answer) and sometimes the answer is transposed and modified (tonal answer). In his paper, Wattenberg offers the possibility of matching melodic intervals (instead of pitch-strings), but this too would not match a subject to its tonal answer. So, how do we formalize the relationship between a subject and tonal answer? Or, the improvisational “licks” of Dizzy Gillespie? Such that we could define “themes” as a category that may be formalized computationally? Especially, when a computer does not know where to look for themes or what modifications to expect? Algorithms informed by humanistic understanding of music work much better than those that are not.

There are three primary components of the EMViz MATLAB Runtime application: (1) Data Preparation (Pre-Processing); (2) Pattern Recognition; and (3) Visualization. These components are compared between Shape of Song, EMViz (now available in MATLAB Runtime, and ATAVizM (currently being developing Python) in Table 1.
There are three primary components of the EMViz MATLAB Runtime application: (1) Data Preparation (Pre-Processing); (2) Pattern Recognition; and (3) Visualization. These components are compared between Shape of Song, EMViz (now available in MATLAB Runtime, and ATAVizM (currently being developed in Python) in Table 1.

![Table of features of Wattenberg's Shape of Song, EMViz (released 2017) and ATAVizM (in development)](table1.png)

The computationally-efficient addition of the contour level data preparation (see Carter-Enyi 2016b) [@Carter-Enyi:2016b] makes equivalence-based pattern recognition (e.g. string-matching) much more effective. Notably, this pre-processing of the symbolic data makes significant improvements to performance with negligible added runtime. ATAVizM’s algorithm identifies 100% of the subjects and approximately 80% of all statements in the 48 Fugues of Bach’s Well-Tempered Clavier.


The algorithm brings together contour theory ([@Morris:1987], [@Quinn:1997]) with studies of melodic accent ([@Thomassen:1982], [@Huron:2006]). Symbolic music data (.midi, .xml) from various sources (including ELVIS at McGill and the Yale Classical Archives Corpus) may be imported, analyzed and visualized in a matter of minutes. The contour algorithm behind EMViz uses local contour comparisons within a 2-degree radius of each focus pitch. This makes use of Quinn’s binary C+ comparison, where 0 is equal or below and 1 is above the reference pitch [@Quinn:1997]. In Figure 1, the subject and tonal answer of Bach’s C-minor Fugue (BWV 847) are shown to be equivalent strings using this method (they both produce the same “continuous” matrix).
The algorithm brings together contour theory [@Morris:1987], [@Quinn:1997] with studies of melodic accent [@Thomassen:1982], [@Huron:2006]. Symbolic music data (.midi, .xml) from various sources (including ELVIS at McGill and the Yale Classical Archives Corpus) may be imported, analyzed and visualized in a matter of minutes. The contour algorithm behind EMViz uses local contour comparisons within a 2-degree radius of each focus pitch. This makes use of Quinn’s binary C+ comparison, where 0 is equal or below and 1 is above the reference pitch [@Quinn:1997]. In Figure 1, the subject and tonal answer of Bach’s C-minor Fugue (BWV 847) are shown to be equivalent strings using this method (they both produce the same “continuous” matrix).

![Contour level equivalence between the subject and tonal answer of Bach's Fugue in C minor BWV 847 based on Carter-Enyi 2016b [@Carter-Enyi:2016b]](figure1.png)

Expand Down

0 comments on commit 7ca1b35

Please sign in to comment.