A General Framework for Visualization of Sound Collections in Musical Interfaces
Gerard Roma [1], Anna Xambó [2], Owen Green [1] and Pierre Alexandre Tremblay [1]
[1] Centre for Research into New Music (CeReNeM), University of Huddersfield
[2] Music, Technology and Innovation (MTI2), De Montfort University
Abstract
While audio data plays an increasingly central role in computer-based music production, interaction with large sound collections in most available music creation and production environments is still limited to scrolling long lists of file names. This paper describes a general framework for devising interactive applications based on content-based visualization of sound collections. The proposed framework allows a modular combination of different techniques for sound segmentation, analysis, and dimensionality reduction, then using the reduced feature space for interactive applications. We analyze several prototypes presented in the literature and describe their limitations. We propose a more general framework that can be used flexibly to devise music creation interfaces. The proposed approach includes several novel contributions with respect to previously used pipelines, such as using unsupervised feature learning, content-based sound icons, and control of the output space layout. We present an implementation of the framework using the SuperCollider computer music language, and three prototypes demonstrating its use for data-driven music interfaces. Our results demonstrate the potential of unsupervised machine learning and visualization for creative applications in computer music.
Examples
Slicer
Slicer Demo from Fluid Corpus Map on Vimeo.
Multislider
Multislider Demo from Fluid Corpus Map on Vimeo.
Trajectories
Trajectories Demo from Fluid Corpus Map on Vimeo.