Visual attention during L1 and L2 sounds perception: an eye-tracking study
DOI:
https://doi.org/10.36505/ExLing-2010/03/0043/000163Keywords:
Audiovisual speech, multisensory integration, native and non-native perceptionAbstract
Visual information affects speech perception as demonstrated by the McGurk effect (McGurk & McDonald, 1976): when audio /ba/ is dubbed with a visual /ga/, what is perceived is /da/. This study aims at observing how visual information, intended as articulatory orofacial movements, is processed by eye, i.e., if gaze is related to articulatory information processing. The results indicate that visual attentional resources seem to be higher during multisensory (AV) than unisensory (A; V) presentation. Probably, higher visual attentional resources are needed to integrate inputs coming from different sources. Moreover, audiovisual speech perception seems to be similar across languages (e.g., Chen & Massaro, 2004) and not language-specific (Ghazanfar et al., 2005).
References
Chen, T., & Massaro, D.W. 2004. Mandarin speech perception by ear and eye follows a universal principle. Perception & Psychophysics, 66(5), 820–836.
Ghazanfar, A.A., Maier, J.X., Hoffman, K.L., & Logothetis, N.K. 2005. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. The Journal of Neuroscience, 25(20), 5004–5012.
Hazan, V., Sennema, A., & Faulkner, A. 2002. Audiovisual perception in L2 learners. In: Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP 2002), 1685–1688. Denver, CO.
McGurk, H., & MacDonald, J. 1976. Hearing lips and seeing voices. Nature, 264(5588), 746–748.
Navarra, J., & Soto-Faraco, S. 2007. Hearing lips in a second language. Psychological Research, 71(1), 4–12.
Tiippana, K., Andersen, T.S., & Sams, M. 2004. Visual attention modulates audiovisual speech perception. European Journal of Cognitive Psychology, 16(3), 457–472.
Downloads
Published
Issue
Section
License
Articles are published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly credited.
