The role of visual spatial attention in audiovisual speech perception

T.S. Andersen (Corresponding Author), K. Tiippana, Jari Laarni, I. Kojo, M. Sams

    Research output: Contribution to journalArticleScientificpeer-review

    27 Citations (Scopus)

    Abstract

    Auditory and visual information is integrated when perceiving speech, as evidenced by the McGurk effect in which viewing an incongruent talking face categorically alters auditory speech perception. Audiovisual integration in speech perception has long been considered automatic and pre-attentive but recent reports have challenged this view. Here we study the effect of visual spatial attention on the McGurk effect. By presenting a movie of two faces symmetrically displaced to each side of a central fixation point and dubbed with a single auditory speech track, we were able to discern the influences from each of the faces and from the voice on the auditory speech percept. We found that directing visual spatial attention towards a face increased the influence of that face on auditory perception. However, the influence of the voice on auditory perception did not change suggesting that audiovisual integration did not change. Visual spatial attention was also able to select between the faces when lip reading. This suggests that visual spatial attention acts at the level of visual speech perception prior to audiovisual integration and that the effect propagates through audiovisual integration to influence auditory perception.
    Original languageEnglish
    Pages (from-to)184-193
    Number of pages10
    JournalSpeech Communication
    Volume51
    Issue number2
    DOIs
    Publication statusPublished - 2009
    MoE publication typeA1 Journal article-refereed

    Fingerprint

    Speech Perception
    Face
    Visual Perception
    Fixation
    movies
    Vision
    Spatial Attention
    Audiovisual Speech
    Influence
    Hearing

    Cite this

    Andersen, T.S. ; Tiippana, K. ; Laarni, Jari ; Kojo, I. ; Sams, M. / The role of visual spatial attention in audiovisual speech perception. In: Speech Communication. 2009 ; Vol. 51, No. 2. pp. 184-193.
    @article{600ca9c4dc1e460c87dd3e627ff5f5d0,
    title = "The role of visual spatial attention in audiovisual speech perception",
    abstract = "Auditory and visual information is integrated when perceiving speech, as evidenced by the McGurk effect in which viewing an incongruent talking face categorically alters auditory speech perception. Audiovisual integration in speech perception has long been considered automatic and pre-attentive but recent reports have challenged this view. Here we study the effect of visual spatial attention on the McGurk effect. By presenting a movie of two faces symmetrically displaced to each side of a central fixation point and dubbed with a single auditory speech track, we were able to discern the influences from each of the faces and from the voice on the auditory speech percept. We found that directing visual spatial attention towards a face increased the influence of that face on auditory perception. However, the influence of the voice on auditory perception did not change suggesting that audiovisual integration did not change. Visual spatial attention was also able to select between the faces when lip reading. This suggests that visual spatial attention acts at the level of visual speech perception prior to audiovisual integration and that the effect propagates through audiovisual integration to influence auditory perception.",
    author = "T.S. Andersen and K. Tiippana and Jari Laarni and I. Kojo and M. Sams",
    year = "2009",
    doi = "10.1016/j.specom.2008.07.004",
    language = "English",
    volume = "51",
    pages = "184--193",
    journal = "Speech Communication",
    issn = "0167-6393",
    publisher = "Elsevier",
    number = "2",

    }

    The role of visual spatial attention in audiovisual speech perception. / Andersen, T.S. (Corresponding Author); Tiippana, K.; Laarni, Jari; Kojo, I.; Sams, M.

    In: Speech Communication, Vol. 51, No. 2, 2009, p. 184-193.

    Research output: Contribution to journalArticleScientificpeer-review

    TY - JOUR

    T1 - The role of visual spatial attention in audiovisual speech perception

    AU - Andersen, T.S.

    AU - Tiippana, K.

    AU - Laarni, Jari

    AU - Kojo, I.

    AU - Sams, M.

    PY - 2009

    Y1 - 2009

    N2 - Auditory and visual information is integrated when perceiving speech, as evidenced by the McGurk effect in which viewing an incongruent talking face categorically alters auditory speech perception. Audiovisual integration in speech perception has long been considered automatic and pre-attentive but recent reports have challenged this view. Here we study the effect of visual spatial attention on the McGurk effect. By presenting a movie of two faces symmetrically displaced to each side of a central fixation point and dubbed with a single auditory speech track, we were able to discern the influences from each of the faces and from the voice on the auditory speech percept. We found that directing visual spatial attention towards a face increased the influence of that face on auditory perception. However, the influence of the voice on auditory perception did not change suggesting that audiovisual integration did not change. Visual spatial attention was also able to select between the faces when lip reading. This suggests that visual spatial attention acts at the level of visual speech perception prior to audiovisual integration and that the effect propagates through audiovisual integration to influence auditory perception.

    AB - Auditory and visual information is integrated when perceiving speech, as evidenced by the McGurk effect in which viewing an incongruent talking face categorically alters auditory speech perception. Audiovisual integration in speech perception has long been considered automatic and pre-attentive but recent reports have challenged this view. Here we study the effect of visual spatial attention on the McGurk effect. By presenting a movie of two faces symmetrically displaced to each side of a central fixation point and dubbed with a single auditory speech track, we were able to discern the influences from each of the faces and from the voice on the auditory speech percept. We found that directing visual spatial attention towards a face increased the influence of that face on auditory perception. However, the influence of the voice on auditory perception did not change suggesting that audiovisual integration did not change. Visual spatial attention was also able to select between the faces when lip reading. This suggests that visual spatial attention acts at the level of visual speech perception prior to audiovisual integration and that the effect propagates through audiovisual integration to influence auditory perception.

    U2 - 10.1016/j.specom.2008.07.004

    DO - 10.1016/j.specom.2008.07.004

    M3 - Article

    VL - 51

    SP - 184

    EP - 193

    JO - Speech Communication

    JF - Speech Communication

    SN - 0167-6393

    IS - 2

    ER -