• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

2013 Fiscal Year Final Research Report

Visual speech recognition using ultrasound tongue and video lip/face images

Research Project

  • PDF
Project/Area Number 23520467
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Linguistics
Research InstitutionThe University of Aizu

Principal Investigator

WILSON Ian  会津大学, コンピュータ理工学部, 教授 (50444930)

Project Period (FY) 2011 – 2013
Keywordsultrasound / video / tongue / articulation / jaw
Research Abstract

There are three main results of our research: (1) Related to video data collection of jaw movement, when measuring the amount of skin stretching over the mandible for the vowel in a CVC syllable, the onset consonant (but not the coda consonant) has a significant effect. (2) Related to ultrasound data collection of tongue position when speaking English, native (L1) speakers rest their tongue in a more efficient location (closer to the median position for English speech sounds) than Japanese (L2) speakers do. (3) Related to our focus on how best to construct and interpret a feature space we call MUTIS (midsagittal ultrasound tongue image space), results indicated that higher dimensions of MUTIS are most effective for identifying people, and that primarily the lower dimensions of VSS (vocal sound space) data are most effective for identifying phonemes. Trajectories within the VSS data indicate clear differences between L1 and L2 speakers, but not within the MUTIS data alone.

  • Research Products

    (14 results)

All 2013 2012 Other

All Journal Article (9 results) Presentation (4 results) Remarks (1 results)

  • [Journal Article] Effect of syllable onset, coda, and nucleus on degree of skin stretching over the mandible2013

    • Author(s)
      Wilson, I. & D.Erickson
    • Journal Title

      Proceedings of Meetings on Acoustics

      Volume: vol.19

    • DOI

      10.1121/1.4799467

  • [Journal Article] Normalization and matching routine for comparing first and second language tongue trajectories2013

    • Author(s)
      Moriya, S., Y.Yaguchi, N.Terunuma, T.Sato, & I.Wilson
    • Journal Title

      Journal of the Acoustical Society of America

      Volume: vol.134, No.5, Pt.2 Pages: 4244

    • DOI

      10.1121/1.4831607

  • [Journal Article] Coarticulatory effects of lateral tongue bracing in first and second language English speakers2013

    • Author(s)
      Kanada, S., I.Wilson, B.Gick, & D.Erickson
    • Journal Title

      Journal of the Acoustical Society of America

      Volume: vol.134, No.5, Pt.2 Pages: 4244

    • DOI

      10.1121/1.4831608

  • [Journal Article] 舌特徴空間における言語学習者の違いを比較するための正規化とマッチング手法2013

    • Author(s)
      Moriya, S., Y. Yaguchi, N. Terunuma, T. Sato, & I. Wilson
    • Journal Title

      IEICE Technical Report

      Volume: vol.113, No.308, SP2013-80 Pages: 53-57

  • [Journal Article] Articulating rhythm in L1 and L2 English : Focus on jaw and F02012

    • Author(s)
      Wilson, I., D.Erickson, & N.Horiguchi
    • Journal Title

      Proceedings of the 2012 Autumn Meeting of the Acoustical Society of Japan (ASJ)

      Pages: 319-322

  • [Journal Article] Finding phoneme trajectories in a feature space of sound and midsagittal ultrasound tongue images2012

    • Author(s)
      Yaguchi, Y., N.Horiguchi, & I.Wilson
    • Journal Title

      In IEEE Proceedings of the 4th International Conference on Awareness Science and Technology (iCAST 2012)

      Pages: 156-162

    • DOI

      10.1109/iCAwST.2012.6469606

  • [Journal Article] Video recordings of L1 and L2 jaw movement : Effect of syllable onset on jaw opening during syllable nucleus2012

    • Author(s)
      Abe, Y., I.L.Wilson, & D.Erickson
    • Journal Title

      Journal of the Acoustical Society of America

      Volume: vol.132, No.3, Pt.2 Pages: 2005

    • DOI

      10.1121/1.4755428

  • [Journal Article] Pitch and intensity in the speech of Japanese speakers of English : Comparison with L1 speakers2012

    • Author(s)
      Okada, J., I.L.Wilson, & M.Yoshizawa
    • Journal Title

      Journal of the Acoustical Society of America

      Volume: vol.132, No.3, Pt.2 Pages: 2004

    • DOI

      10.1121/1.4755421

  • [Journal Article] Comparing L1 and L2 phoneme trajectories in a feature space of sound and midsagittal ultrasound tongue images2012

    • Author(s)
      Sano, K., Y.Yaguchi, & I.Wilson
    • Journal Title

      Journal of the Acoustical Society of America

      Volume: vol.132, No.3, Pt.2 Pages: 1934

    • DOI

      10.1121/1.4755107

  • [Presentation] Lateral tongue bracing in Japanese and English2013

    • Author(s)
      Wilson, I., J.Villegas, & T.Doi
    • Organizer
      Paper presented at Ultrafest VI
    • Place of Presentation
      Edinburgh, Scotland
    • Year and Date
      2013-11-08
  • [Presentation] Articulatory and laryngeal contributions to rhythm in English2013

    • Author(s)
      Erickson, D. & I.Wilson
    • Organizer
      the Joint Research Meeting of the Dept. of Linguistic Theory and Structur
    • Place of Presentation
      NINJAL, Tokyo, Japan (Poster presented)
    • Year and Date
      2013-03-02
  • [Presentation] How accurately people follow articulation instructions2012

    • Author(s)
      Wilson, I. & N.Horiguchi
    • Organizer
      Paper presented at the 4th Pronunciation in Second Language Learning and Teaching conference (PSLLT 2012)
    • Place of Presentation
      Vancouver, Canada
    • Year and Date
      2012-08-24
  • [Presentation] 発音習得のための超音波舌画像に対する音素片マッピング [Mapping phonemes to midsagittal tongue images for pronunciation learning]2012

    • Author(s)
      Yaguchi, Y., N.Horiguchi, & I.Wilson
    • Organizer
      the joint meeting of the Technical Committees for Pattern Recognition and Media Understanding (PRMU) and Signal Processing (SP) of the Institute of Electronics, Information and Communication Engineers (IEICE)
    • Place of Presentation
      Sendai, Japan (Paper presented)
    • Year and Date
      2012-02-10
  • [Remarks]

    • URL

      http://clrlab1.u-aizu.ac.jp/index_j.html

URL: 

Published: 2015-07-16   Modified: 2021-04-07  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi