Towards note-level prediction for networked music performance

Reid Oda, Adam Finkelstein, Rebecca Fiebrink

Research output: Contribution to journalConference articlepeer-review

6 Scopus citations

Abstract

The Internet allows musicians and other artists to collaborate remotely. However, network latency presents a fundamental challenge for remote collaborators who need to coordinate and respond to each other’s performance in real time. In this paper, we investigate the viability of predicting percussion hits before they have occurred, so that information about the predicted drum hit can be sent over a network, and the sound can be synthesized at a receiver’s location at approximately the same moment the hit occurs at the sender’s location. Such a system would allow two percussionists to play in perfect synchrony despite the delays caused by computer networks. To investigate the feasibility of such an approach, we record vibraphone mallet strikes with a high-speed camera and track the mallet head position. We show that 30 ms before the strike occurs, it is possible to predict strike time and velocity with acceptable accuracy. Our method fits a second-order polynomial to the data to produce a strike-time prediction that is within 10 ms of the actual strike, and a velocity estimate that will enable the sound pressure level of the synthesized strike to be accurate within 3 dB.

Original languageEnglish (US)
Pages (from-to)94-97
Number of pages4
JournalProceedings of the International Conference on New Interfaces for Musical Expression
StatePublished - 2013
Externally publishedYes
Event13th International conference on New Interfaces for Musical Expression, NIME 2013 - Daejeon, Korea, Republic of
Duration: May 27 2013May 30 2013

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Signal Processing
  • Instrumentation
  • Music
  • Human-Computer Interaction
  • Hardware and Architecture
  • Computer Science Applications

Keywords

  • Computer vision
  • Networked performance
  • Prediction

Fingerprint

Dive into the research topics of 'Towards note-level prediction for networked music performance'. Together they form a unique fingerprint.

Cite this