• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of high-definition audio-visual speech communication systems based on the knowledge of /kansei/ information processing

Research Project

Project/Area Number 23500252
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Sensitivity informatics/Soft computing
Research InstitutionTohoku University

Principal Investigator

SAKAMOTO Shuichi  東北大学, 電気通信研究所, 准教授 (60332524)

Project Period (FY) 2011 – 2013
Project Status Completed (Fiscal Year 2013)
Budget Amount *help
¥5,200,000 (Direct Cost: ¥4,000,000、Indirect Cost: ¥1,200,000)
Fiscal Year 2013: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2012: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2011: ¥2,990,000 (Direct Cost: ¥2,300,000、Indirect Cost: ¥690,000)
Keywords感性情報学 / 情報システム / 音声認知 / 視聴覚コミュニケーション / 視聴覚情報処理
Research Abstract

It is important to transmit talker's mind including emotion for developing advanced communication system. Because not only speech signal itself but also moving image of talkers face includes rich information to understand what talker wants to tell, these information should be transmit accurately to the listeners. However, the relationship between the physical parameter extracted from audio-visual speech signal and /kansei/ information is unclear.
The final goal of this study is to develop high-definition audio-visual speech communication systems. To do this, the effect of moving image of talker's face to speech understanding was investigated. The results of psychophysical experiments indicate that around the lips area is crucial to understand what talker says. Moreover, /kansei/ impression from audio-visual speech signal is also important for speech understanding.

Report

(4 results)
  • 2013 Annual Research Report   Final Research Report ( PDF )
  • 2012 Research-status Report
  • 2011 Research-status Report
  • Research Products

    (19 results)

All 2014 2013 2012

All Journal Article (9 results) (of which Peer Reviewed: 3 results) Presentation (10 results) (of which Invited: 2 results)

  • [Journal Article] The contribution of the detailed parts around talker's mouth for speech intelligibility2014

    • Author(s)
      S. Sakamoto, G. Hasegawa, T. Abe, T. Ohtani, Y. Suzuki and K. Kawase
    • Journal Title

      Proc. The 21st International Congress on Sound and Vibration (ICSV21)

      Pages: 7-7

    • Related Report
      2013 Final Research Report
    • Peer Reviewed
  • [Journal Article] 口唇以外の話者映像情報が無意味 3連音節を用いた音声明瞭度に与える影響2014

    • Author(s)
      長谷川玄, 坂本修一, 阿部亨, 大谷智子, 鈴木陽一, 川瀬哲明
    • Journal Title

      日本音響学会講演論文集

      Volume: 2-P5-21 Pages: 641-642

    • Related Report
      2013 Final Research Report
  • [Journal Article] 口唇以外の話者映像情報が無意味3連音節を用いた音声明瞭度に与える影響2014

    • Author(s)
      長谷川玄,坂本修一,阿部亨,大谷智子,鈴木陽一,川瀬哲明
    • Journal Title

      日本音響学会講演論文集

      Volume: 2-P5-21 Pages: 641-642

    • Related Report
      2013 Annual Research Report
  • [Journal Article] The contribution of the detailed parts around talker's mouth for speech intelligibili2014

    • Author(s)
      Shuichi Sakamoto, Gen Hasegawa, Toru Abe, Tomoko Ohtani, Yo-iti Suzuki and Tetsuaki Kawase
    • Journal Title

      Proc. the 21st International Congress on Sound and Vibration (ICSV21)

      Volume: -

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 無意味 3連音節を用いた音素別明瞭度における話者映像の寄与の分析2013

    • Author(s)
      長谷川玄, 坂本修一, 阿部亨, 大谷智子, 鈴木陽一, 川瀬哲明
    • Journal Title

      電子情報通信学会技術研究報告

      Volume: HIP2013-60 Pages: 1-6

    • Related Report
      2013 Final Research Report
  • [Journal Article] 無意味 3連音節を用いた音素別明瞭度における視覚情報の寄与の分析2013

    • Author(s)
      長谷川玄, 坂本修一, 阿部亨, 大谷智子, 鈴木陽一, 川瀬哲明
    • Journal Title

      日本音響学会聴覚研究会資料

      Volume: H-2013-102 Pages: 595-600

    • NAID

      40019839445

    • Related Report
      2013 Final Research Report
  • [Journal Article] 無意味3連音節を用いた音素別明瞭度における視覚情報の寄与の分析2013

    • Author(s)
      長谷川玄,坂本修一,阿部亨,大谷智子,鈴木陽一,川瀬哲明
    • Journal Title

      日本音響学会聴覚研究会資料

      Volume: H-2013-102 Pages: 595-600

    • NAID

      40019839445

    • Related Report
      2013 Annual Research Report
  • [Journal Article] 無意味3連音節を用いた音素別明瞭度における話者映像の寄与の分析2013

    • Author(s)
      長谷川玄,坂本修一,阿部亨,大谷智子,鈴木陽一,川瀬哲明
    • Journal Title

      電子情報通信学会技術研究報告

      Volume: HIP2013-60 Pages: 1-6

    • Related Report
      2013 Annual Research Report
  • [Journal Article] Effect of consonance between features and voice impression on the McGurk effect2012

    • Author(s)
      S.Sakamoto, H. Mishima and Y. Suzuki
    • Journal Title

      Interdisciplinary Information Sciences Journal

      Volume: 18 Pages: 83-85

    • Related Report
      2013 Final Research Report 2012 Research-status Report
    • Peer Reviewed
  • [Presentation] The contribution of the detailed parts around talker's mouth for speech intelligibility2014

    • Author(s)
      S. Sakamoto, G. Hasegawa, T. Abe, T. Ohtani, Y. Suzuki and K. Kawase
    • Organizer
      Proc. The 21st International Congress on Sound and Vibration (ICSV21)
    • Place of Presentation
      Beijing, China
    • Related Report
      2013 Final Research Report
    • Invited
  • [Presentation] Contribution of detailed parts around talker's mouth for audio-visual speech perception2014

    • Author(s)
      S. Sakamoto, G. Hasegawa, T. Abe, T. Ohtani, Y. Suzuki and K. Kawase
    • Organizer
      167th Meeting of the Acoustical Society of America
    • Place of Presentation
      Providence, USA
    • Related Report
      2013 Final Research Report
  • [Presentation] 口唇以外の話者映像情報が無意味 3連音節を用いた音声明瞭度に与える影響2014

    • Author(s)
      長谷川玄, 坂本修一, 阿部亨, 大谷智子, 鈴木陽一, 川瀬哲明
    • Organizer
      日本音響学会 2014年春季研究発表会
    • Place of Presentation
      日本大学
    • Related Report
      2013 Final Research Report
  • [Presentation] 口唇以外の話者映像情報が無意味3連音節を用いた音声明瞭度に与える影響2014

    • Author(s)
      長谷川玄
    • Organizer
      日本音響学会2014年春季研究発表会
    • Place of Presentation
      日本大学
    • Related Report
      2013 Annual Research Report
  • [Presentation] Contribution of detailed parts around talker's mouth for audio-visual speech perception2014

    • Author(s)
      Shuichi Sakamoto
    • Organizer
      167th Meeting of the Acoustical Society of America
    • Place of Presentation
      Providence, RI, USA
    • Related Report
      2013 Annual Research Report
  • [Presentation] 無意味 3連音節を用いた音素別明瞭度における話者映像の寄与の分析2013

    • Author(s)
      長谷川玄, 坂本修一, 阿部亨, 大谷智子, 鈴木陽一, 川瀬哲明
    • Organizer
      電子情報通信学会ヒューマン情報処理(HIP)研究会
    • Place of Presentation
      東北大学
    • Related Report
      2013 Final Research Report
  • [Presentation] 無意味 3連音節を用いた音素別明瞭度における視覚情報の寄与の分析2013

    • Author(s)
      長谷川玄, 坂本修一, 阿部亨, 大谷智子, 鈴木陽一, 川瀬哲明
    • Organizer
      日本音響学会聴覚研究会
    • Place of Presentation
      神戸セミナーハウス
    • Related Report
      2013 Final Research Report
  • [Presentation] 無意味3連音節を用いた音素別明瞭度における視覚情報の寄与の分析2013

    • Author(s)
      長谷川玄
    • Organizer
      日本音響学会聴覚研究会
    • Place of Presentation
      神戸セミナーハウス
    • Related Report
      2013 Annual Research Report
  • [Presentation] 無意味3連音節を用いた音素別明瞭度における話者映像の寄与の分析2013

    • Author(s)
      長谷川玄
    • Organizer
      電子情報通信学会ヒューマン情報処理(HIP)研究会
    • Place of Presentation
      東北大学電気通信研究所
    • Related Report
      2013 Annual Research Report
  • [Presentation] The contribution of the detailed parts around talker's mouth for speech intelligibility2013

    • Author(s)
      Shuichi Sakamoto
    • Organizer
      the 21st International Congress on Sound and Vibration (ICSV21)
    • Place of Presentation
      Beijing, China
    • Related Report
      2013 Annual Research Report
    • Invited

URL: 

Published: 2011-08-05   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi