• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

SPEECH RECOGNITION WITH SYNCHRONOUS INPUT OF HAND-WRITTEN GESTURES FOR MOBILE DEVICES

Research Project

Project/Area Number 15300054
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Perception information processing/Intelligent robotics
Research InstitutionTokyo Institute of Technology

Principal Investigator

SHINODA Koichi  Tokyo Institute of Technology, Graduate School of Information science and engineering, Associate Professor, 情報理工学研究科, 助教授 (10343097)

Co-Investigator(Kenkyū-buntansha) FURUI Sadaoki  Tokyo Institute of Technology, Information science and engineering, Professor, 情報理工学研究科, 教授 (90293076)
Project Period (FY) 2003 – 2004
Project Status Completed (Fiscal Year 2004)
Budget Amount *help
¥5,900,000 (Direct Cost: ¥5,900,000)
Fiscal Year 2004: ¥3,100,000 (Direct Cost: ¥3,100,000)
Fiscal Year 2003: ¥2,800,000 (Direct Cost: ¥2,800,000)
KeywordsSpeech Recognition / Multimodal Interface / Hand-written Character Recognition / Search Algorithm / Man-Machine Interface / Mobile Device / Hiddne Markov MOdel / 情報携帯端末 / 携帯情報端末
Research Abstract

Mobile devices have recently been often used in daily life. User-friendly interface with high accuracy has been strongly demanded. For this purpose, we propose an interface using simultaneous inputs of speech and hand-written gestures. This interface is more robust against environmental noise than speech-only interface, and its input speed is faster than the interface with only hand-written gestures. Our target application is e-mail making with the input of sentences.
First year, we proposed an interface in which a sentence is input by speech while the "hiragana" character at the head of each phrase in the sentence is input by hand-written gestures. We implemented a recognition algorithm for hand-written gestures, designed a method for recognizing the simultaneous inputs of the two modes. The proposed method was evaluated by simulation experiments using speech data and hand-written gesture data, which are recorded independently, and was proved to be effective.
Second year, we constructed a recording system for the input of the two modes, and recorded 530 sentences from ten subjects. For integrating the two modes, we employed a two-pass process in which a word graph generated by speech recognition in the first pass is utilized for the integration process of the two modes in the second pass. The proposed method improved the recognition accuracy by 2.6 point over the method only with speech recognition.
For future work, a method for optimizing the weights among the two modes should be developed. We are going to develop a demonstration system which works in real time and evaluate it in noisy environment.

Report

(3 results)
  • 2004 Annual Research Report   Final Research Report Summary
  • 2003 Annual Research Report
  • Research Products

    (5 results)

All 2005 2004 Other

All Journal Article (4 results) Publications (1 results)

  • [Journal Article] 音声と手書き文字の同時入力インターフェース2005

    • Author(s)
      中川, 小林(唯), 小林(隆), 篠田, 古井
    • Journal Title

      2005年電子情報通信学会技術研究報告 2005年5月(印刷中)

    • NAID

      110002950654

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Simultaneous Input Interface of Speech and Handwritten Characters2005

    • Author(s)
      R.Nakagawa, Y.Kobayashi, R.Kobayashi, K.Shinoda, S.Furui
    • Journal Title

      Technical Report of IEICE vol.SP2005-5(in press)

    • NAID

      110002950654

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 手書き文字の準同期入力を併用した音声認識手法の予備検討2004

    • Author(s)
      市屋, 中川, 篠田, 古井
    • Journal Title

      2004年電子情報通信学会総合大会講演論文集 D

      Pages: 148-148

    • NAID

      110003263600

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Annual Research Report 2004 Final Research Report Summary
  • [Journal Article] Preliminary Evaluation of Speech Recognition with Quasi-Synchronous Input of Hand-Written Characters.2004

    • Author(s)
      T.Ichiya, R.Nakagawa, K.Shinoda, S.Furui
    • Journal Title

      Proc.Of 2004 IEICE General Conference vol.D

      Pages: 148-148

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Publications] 市屋, 中川, 篠田, 古井: "手書き文字の準同期入力を併用した音声認識手法の予備検討"2004年電子情報通信学会総合大会講演論文集. D. 148-148 (2004)

    • Related Report
      2003 Annual Research Report

URL: 

Published: 2003-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi