• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development in auditory-visual speech perception

Research Project

Project/Area Number 10610070
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeSingle-year Grants
Section一般
Research Field 実験系心理学
Research InstitutionKanazawa University

Principal Investigator

SEKIYAMA Kaoru  Kanazawa Univ., Faculty of Letters, Instructor, 文学部, 助手 (70216539)

Project Period (FY) 1998 – 1999
Project Status Completed (Fiscal Year 1999)
Budget Amount *help
¥3,400,000 (Direct Cost: ¥3,400,000)
Fiscal Year 1999: ¥500,000 (Direct Cost: ¥500,000)
Fiscal Year 1998: ¥2,900,000 (Direct Cost: ¥2,900,000)
Keywordsauditory-visual / audiovisual speech perception / audiovisual integration / McGurk effect / development / 読唇 / 視聴覚統合
Research Abstract

Inter-sensory integration has been investigated especially in terms of the nature-nurture problem. The present study examined developmental changes in auditory-visual integration in speech perception including the McGurk effect. The McGurk effect is an audiovisual illusion showing that hearing speech is influenced by conflicting visual lip-read information.
The subjects Were 3-year-old, 7-year-old, 11-year-old, and 20-year-old native speakers of Japanese. Each age group included 10 subjects. Stimuli were created from/ba/and/da/spoken by a Japanese female talker. Videotaped syllables were edited, resulting in audiovisually conflicting stimuli (audio/ba/, video/da/, and vice versa) as well as audiovisually matching stimuli (audio/ba/, video/ba/). In addition to original intact stimuli, degraded stimuli were prepared. The degraded auditory stimuli were created by lowpass filtering with a cut-off frequency of 730 kHz, the degraded visual stimuli were obtained by mosaic effect at and around … More the mouth. Based on pilot experiments, the level of the degradation was set so that accuracy of the unimodal performance was about 80% in the 20-year-old. There were several modality conditions : A-only (clear and lowpass), V-only (clear and mosaic), and AV (combinations of the unimodal conditions). The task of the subjects were to report what they perceived by choosing either/ba/or/da/.
In the-A- and V-only conditions, clear stimuli were perceived very accurately, showing no age differences. In the AV conditions, the size of the McGurk effect was the same across the age groups when the auditory component was clear. When the auditory component was degraded, the size of the McGurk effect was significantly smaller in the 3-year-old than in the other age groups, although the 3-year-old showed as strong a McGurk effect as the other age groups.
These results suggest that the integration for audiovisually conflicting stimuli emerges relatively late, whereas audiovisually matching stimuli are perceptually integrated at a very early stage of life. Less

Report

(3 results)
  • 1999 Annual Research Report   Final Research Report Summary
  • 1998 Annual Research Report
  • Research Products

    (24 results)

All Other

All Publications (24 results)

  • [Publications] 積山 薫: "雑音下の視聴覚音声知覚―調音位置による違い―"電子情報通信学会技術研究報告. SP99-51. 1-8 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 林康子、積山薫: "音素カテゴリーの発達―劣化した音声による検討―"電子情報通信学会技術研究報告. SP99-44. 1-8 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 積山薫・L.D.ブライダ、D.A.ディックス: "単一モダリティ条件からのマガーク効果の予測"日本音響学会聴覚研究会資料. H-98-94. 1-8 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Sekiyama, K.: "Face or voice? Determinant of compellingness to the McGurk effect"Proceedings of AVSP'98 (Burnham et al.(Eds.)). 33-36 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Amano, J. & Sekiyama, K.: "The McGurk effect is influenced by the stimulus set size"Proceedings of AVSP'98 (Burnham et al.(Eds.)). 43-48 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Braida, L. D., Sekiyama, K. & Dix, A.: "Integration of audiovisually compatible and incompatible consonants in identification experiments"Proceedings of AVSP'98 (Burnham et al.(Eds.)). 49-54 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Sekiyama, K.: "Audiovisual speech perception under various signal-to-noise ratios : Asymmetry for different places of articulation. (in English)"Technical Report of The Institute of Electronics, Information, and Communication Engineers. SP99-51. 1-8 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Hayashi, Y. & Sekiyama, K.: "Development of phonetic categories : A test with degraded speech. (in Japanese with English abstract)"Technical Report of The Institute of Electronics, Information, and Communication Engineers. SP99-44. 1-8 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Sekiyama, K., Braida, L. D. & Dix, A.: "Predicting the McGurk effect from unimodal performances. (in Japanese with English abstract)"Transaction of the Acoustical Society of Japan. H-98-94. 1-8 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Sekiyama, K.: "Face or voice? Determinant of compellingness to the McGurk effect. (in English)"Proceedings of International Conference on Auditory-Visual Speech Processing '98 (Edited by Burnham, D. et al). 33-36 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Amano, J. & Sekiyama, K.: "The McGurk effect is influenced by the stimulus set size. (in English)"Proceedings of International Conference on Auditory-Visual Speech Processing '98 (Edited by Burnham, D. et al). 43-48 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Braida, L. D., Sekiyama, K. & Dix, A.: "Integration of audiovisually compatible and incompaatible consonant in identification experiments. (in English)"Proceedings of International Conference on Auditory-Visual Speech Processing '98 (Edited by Burnham, D. et al). 49-54 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 積山薫: "雑音下の視聴覚音声視覚-調音位置による違い-"電子情報通信学会技術研究報告. SP99-51. 1-8 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] 林康子,積山薫: "音素カテゴリーの発達-劣化した音声による検討-"電子情報通信学会研究報告. SP99-44. 1-8 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] 積山薫,L.D.ブライダ,D.A.ディックス: "単一モダリティ条件からのマガーク効果の予測"日本音響学会聴覚研究会資料. H-98-94. 1-8 (1998)

    • Related Report
      1999 Annual Research Report
  • [Publications] Sekiyama,K: "Face or voice? Determinant of compellingness to the McGurk effect."Proceedings of AVSP'98.(Burnham,et al. (Eds.)). 33-36 (1998)

    • Related Report
      1999 Annual Research Report
  • [Publications] Amano,J.& Sekiyama,K.: "The McGurk effects is influenced by the stimulus set size."Proceedings of AVSP'98 (Burham,et al. (Eds.)). 43-48 (1998)

    • Related Report
      1999 Annual Research Report
  • [Publications] Brada,L.D.,Sekiyama,K. & Dix,A.: "Integration of audiovisually compatible and incompatible consonants in identification experiments."Proceedings of AVSP'98(Burnham,et al. (Eds.)). 49-54 (1998)

    • Related Report
      1999 Annual Research Report
  • [Publications] 積山 薫: "視覚と聴覚の接点" 日本音響学会誌. 54. 450-456 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] 天野 旬・積山 薫: "刺激セット内の聴覚弁別次元数がマガーク効果に与える影響" 電子情報通信学会技術研究報告. SP98-44. 1-8 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] 積山 薫・L.D.プライダ・D.A.ディックス: "単一モダリティ条件からのマガーク効果の予測" 日本音響学会聴覚研究会資料. H-98-94. 1-8 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] 林 康子・積山 薫: "中国人と日本人のマガーク効果" 日本音響学会聴覚研究会資料. H-98-93. 1-8 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] Sekiyama, K.: "Face or voice? Determinant of compellingness to the McGurk effect." Proceedings of AVSP'98 (Burnham, et al (Ed).). 33-36 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] Amano, J.& Sekiyama, K.: "The McGurK effect is influenced by the stimulus set size." Proceedings of AVSP'98 (Burnham et.al (Ed.)). 43-48 (1998)

    • Related Report
      1998 Annual Research Report

URL: 

Published: 1998-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi