Relationship between paralinguistic information and head motion/facial expression

2010 Fiscal Year Final Research Report

• Project/Area Number: 20680011
• Research Category: Grant-in-Aid for Young Scientists (A)
• Allocation Type: Single-year Grants
• Research Field: Perception information processing/Intelligent robotics
• Research Institution: Advanced Telecommunications Research Institute International
• Principal Investigator: ISHI CarlosToshinori Advanced Telecommunications Research Institute International, 知能ロボティクス研究所, 研究員 (30418529)
• Project Period (FY): 2008 – 2010
• Keywords: 音声情報処理 / 非言語情報

Research Abstract

The present research aimed on clarifying the relationship between speaking styles including prosodic and voice quality features, paralinguistic information such as dialogue acts, intentions and emotions, and visual modalities like head motion and facial expression. As main results, we clarified the paralinguistic functions carried by breathy/whispery voices and "rikimi" voices, we clarified the relations between dialogue acts and head motion, and proposed and evaluated a model for generating nod and head tilt motion from the dialogue acts.

Research Products

• [Journal Article] Analysis of the roles and the dynamics of breathy and whispery voice qualities in dialogue speech. (2010)
  • Author(s): Ishi, C.T., Ishiguro, H., Hagita, N.
  • Journal Title: EURASIP Journal on Audio, Speech, and Music Processing 2010 ID 528193 Pages: 1-12
• [Presentation] Acoustic, electroglottographic and paralinguistic analyses of "rikimi" in expressive speech (2010)
  • Author(s): Ishi, C., Ishiguro, H., Hagita, N.
  • Organizer: Proceedings of Speech Prosody 2010 (SP2010)(ID 100139, 1-4)
  • Year and Date: 20100000
• [Presentation] Head motion during dialogue speech and nod timing control in humanoid robots (2010)
  • Author(s): Ishi, C.T., Liu, C., Ishiguro, H., Hagita, N.
  • Organizer: Proceedings of IEEE/RSJ Human Robot Interaction (HRI 2010)(293-300)
  • Year and Date: 20100000
• [Presentation] Analysis of inter- and intra-speaker variability of head motions during spoken dialogue (2008)
  • Author(s): Ishi, C.T., Ishiguro, H., Hagita, N.
  • Organizer: Proceedings of the International Conference on Auditory-Visual Speech Processing 2008 (AVSP' 2008)(37-42)
  • Year and Date: 20080000
• [Presentation] The meanings of interjections in spontaneous speech (2008)
  • Author(s): Ishi, C.T., Ishiguro, H., Hagita, N.
  • Organizer: Proceedings of The 9th Annual Conference of the International Speech Communication Association (Interspeech' 2008)(1208-1211)
  • Year and Date: 20080000
• [Presentation] The roles of breathy/whispery voice qualities in dialogue speech (2008)
  • Author(s): Ishi, C.T., Ishiguro, H., Hagita, N.
  • Organizer: Proceedings of Speech Prosody 2008(45-48)
  • Year and Date: 20080000