Facilitation of multisensory integration by the "unity effect" reveals that speech is special

Argiro Vatakis, Asif A. Ghazanfar, Charles Spence

Research output: Contribution to journalArticlepeer-review

75 Scopus citations

Abstract

Whenever two or more sensory inputs are highly consistent in one or more dimension(s), observers will be more likely to perceive them as a single multisensory event rather than as separate unimodal events. For audiovisual speech, but not for other noncommunicative events, participants exhibit a "unity effect," whereby they are less sensitive to temporal asynchrony (i.e., that are more likely to bind the multisensory signals together) for matched (than for mismatched) speech events. This finding suggests that the modulation of multisensory integration by the unity effect in humans may be specific to speech. To test this hypothesis directly, we investigated whether the unity effect would also influence the multisensory integration of vocalizations from another primate species, the rhesus monkey. Human participants made temporal order judgments for both matched and mismatched audiovisual stimuli presented at a range of stimulus-onset asynchronies. The unity effect was examined with (1) a single call-type across two different monkeys, (2) two different call-types from the same monkey, (3) human versus monkey "cooing," and (4) speech sounds produced by a male and a female human. The results show that the unity effect only influenced participants' performance for the speech stimuli; no effect was observed for monkey vocalizations or for the human imitations of monkey calls. These findings suggest that the facilitation of multisensory integration by the unity effect is specific to human speech signals.

Original languageEnglish (US)
Article number14
JournalJournal of vision
Volume8
Issue number9
DOIs
StatePublished - Jul 29 2008

All Science Journal Classification (ASJC) codes

  • Sensory Systems
  • Ophthalmology

Keywords

  • Cross-modal
  • Monkey vocalizations
  • Multisensory
  • Temporal perception

Fingerprint

Dive into the research topics of 'Facilitation of multisensory integration by the "unity effect" reveals that speech is special'. Together they form a unique fingerprint.

Cite this