The role of visual spatial attention in audiovisual speech perception

T.S. Andersen (Corresponding Author), K. Tiippana, Jari Laarni, I. Kojo, M. Sams

Research output: Contribution to journalArticleScientificpeer-review

27 Citations (Scopus)

Abstract

Auditory and visual information is integrated when perceiving speech, as evidenced by the McGurk effect in which viewing an incongruent talking face categorically alters auditory speech perception. Audiovisual integration in speech perception has long been considered automatic and pre-attentive but recent reports have challenged this view. Here we study the effect of visual spatial attention on the McGurk effect. By presenting a movie of two faces symmetrically displaced to each side of a central fixation point and dubbed with a single auditory speech track, we were able to discern the influences from each of the faces and from the voice on the auditory speech percept. We found that directing visual spatial attention towards a face increased the influence of that face on auditory perception. However, the influence of the voice on auditory perception did not change suggesting that audiovisual integration did not change. Visual spatial attention was also able to select between the faces when lip reading. This suggests that visual spatial attention acts at the level of visual speech perception prior to audiovisual integration and that the effect propagates through audiovisual integration to influence auditory perception.
Original languageEnglish
Pages (from-to)184-193
Number of pages10
JournalSpeech Communication
Volume51
Issue number2
DOIs
Publication statusPublished - 2009
MoE publication typeA1 Journal article-refereed

Fingerprint

Speech Perception
Face
Visual Perception
Fixation
movies
Vision
Spatial Attention
Audiovisual Speech
Influence
Hearing

Cite this

Andersen, T.S. ; Tiippana, K. ; Laarni, Jari ; Kojo, I. ; Sams, M. / The role of visual spatial attention in audiovisual speech perception. In: Speech Communication. 2009 ; Vol. 51, No. 2. pp. 184-193.
@article{600ca9c4dc1e460c87dd3e627ff5f5d0,
title = "The role of visual spatial attention in audiovisual speech perception",
abstract = "Auditory and visual information is integrated when perceiving speech, as evidenced by the McGurk effect in which viewing an incongruent talking face categorically alters auditory speech perception. Audiovisual integration in speech perception has long been considered automatic and pre-attentive but recent reports have challenged this view. Here we study the effect of visual spatial attention on the McGurk effect. By presenting a movie of two faces symmetrically displaced to each side of a central fixation point and dubbed with a single auditory speech track, we were able to discern the influences from each of the faces and from the voice on the auditory speech percept. We found that directing visual spatial attention towards a face increased the influence of that face on auditory perception. However, the influence of the voice on auditory perception did not change suggesting that audiovisual integration did not change. Visual spatial attention was also able to select between the faces when lip reading. This suggests that visual spatial attention acts at the level of visual speech perception prior to audiovisual integration and that the effect propagates through audiovisual integration to influence auditory perception.",
author = "T.S. Andersen and K. Tiippana and Jari Laarni and I. Kojo and M. Sams",
year = "2009",
doi = "10.1016/j.specom.2008.07.004",
language = "English",
volume = "51",
pages = "184--193",
journal = "Speech Communication",
issn = "0167-6393",
publisher = "Elsevier",
number = "2",

}

The role of visual spatial attention in audiovisual speech perception. / Andersen, T.S. (Corresponding Author); Tiippana, K.; Laarni, Jari; Kojo, I.; Sams, M.

In: Speech Communication, Vol. 51, No. 2, 2009, p. 184-193.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - The role of visual spatial attention in audiovisual speech perception

AU - Andersen, T.S.

AU - Tiippana, K.

AU - Laarni, Jari

AU - Kojo, I.

AU - Sams, M.

PY - 2009

Y1 - 2009

N2 - Auditory and visual information is integrated when perceiving speech, as evidenced by the McGurk effect in which viewing an incongruent talking face categorically alters auditory speech perception. Audiovisual integration in speech perception has long been considered automatic and pre-attentive but recent reports have challenged this view. Here we study the effect of visual spatial attention on the McGurk effect. By presenting a movie of two faces symmetrically displaced to each side of a central fixation point and dubbed with a single auditory speech track, we were able to discern the influences from each of the faces and from the voice on the auditory speech percept. We found that directing visual spatial attention towards a face increased the influence of that face on auditory perception. However, the influence of the voice on auditory perception did not change suggesting that audiovisual integration did not change. Visual spatial attention was also able to select between the faces when lip reading. This suggests that visual spatial attention acts at the level of visual speech perception prior to audiovisual integration and that the effect propagates through audiovisual integration to influence auditory perception.

AB - Auditory and visual information is integrated when perceiving speech, as evidenced by the McGurk effect in which viewing an incongruent talking face categorically alters auditory speech perception. Audiovisual integration in speech perception has long been considered automatic and pre-attentive but recent reports have challenged this view. Here we study the effect of visual spatial attention on the McGurk effect. By presenting a movie of two faces symmetrically displaced to each side of a central fixation point and dubbed with a single auditory speech track, we were able to discern the influences from each of the faces and from the voice on the auditory speech percept. We found that directing visual spatial attention towards a face increased the influence of that face on auditory perception. However, the influence of the voice on auditory perception did not change suggesting that audiovisual integration did not change. Visual spatial attention was also able to select between the faces when lip reading. This suggests that visual spatial attention acts at the level of visual speech perception prior to audiovisual integration and that the effect propagates through audiovisual integration to influence auditory perception.

U2 - 10.1016/j.specom.2008.07.004

DO - 10.1016/j.specom.2008.07.004

M3 - Article

VL - 51

SP - 184

EP - 193

JO - Speech Communication

JF - Speech Communication

SN - 0167-6393

IS - 2

ER -