• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

On The Use of Non-Verbal Speech Sounds in Human Communication

Research Project

Project/Area Number 24500256
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Sensitivity informatics/Soft computing
Research InstitutionNara Institute of Science and Technology

Principal Investigator

CAMPBELL Nick  奈良先端科学技術大学院大学, 情報科学研究科, 教授 (50395109)

Co-Investigator(Kenkyū-buntansha) SADANOBU Toshiyuki  神戸大学, 国際文化学部, 教授 (50235305)
Project Period (FY) 2012-04-01 – 2015-03-31
Project Status Completed (Fiscal Year 2014)
Budget Amount *help
¥5,330,000 (Direct Cost: ¥4,100,000、Indirect Cost: ¥1,230,000)
Fiscal Year 2014: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2013: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Fiscal Year 2012: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000)
Keywordsノンバーバル / 音声発話 / 情動情報 / センシング / 技術 / ヒューマンインターフェース技術 / 非言語コミュニケーション / 感性情報処理 / 音声合成 / マルチモーダルインターフェース / バイオメトリックセンシング / 自律神経系
Outline of Final Research Achievements

This study focused on the processing of non-verbal information to facilitate the transfer of speaker intentions that underlie the spoken text of a social conversational interaction.The contents of natural dialogue, while conveying the semantic meaning of the utterance, also contribute greatly to social ties and informal bonding.
To be aware of such social exchanges, it is essential to process voice information and speaking style (perhaps more important than what was directly said in the conversation).We developed audio processing, alongside speech content information, that is targeted at achieving a more likely interpretation and understanding of speech-interaction.
Prediction of the intentions of the speaker varies depending on the circumstances and the history of the conversation, but reliable paralinguistic signals allow us to develop a system facilitating the interpretation of nonverbal information speech, and to moderate the words according to this extra non-verbal information.

Report

(4 results)
  • 2014 Annual Research Report   Final Research Report ( PDF )
  • 2013 Research-status Report
  • 2012 Research-status Report
  • Research Products

    (37 results)

All 2015 2014 2013 2012 Other

All Journal Article (7 results) (of which Peer Reviewed: 2 results) Presentation (21 results) (of which Invited: 3 results) Book (9 results)

  • [Journal Article] Time for Laughter2014

    • Author(s)
      Francesca Bonin, Nick Campbell, Carl Vogel
    • Journal Title

      Knowledge-based systems

      Volume: 71 Pages: 15-24

    • DOI

      10.1016/j.knosys.2014.04.031

    • Related Report
      2014 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Investigating automatic measurements of prosodic accommodation and its dynamics in social interaction2013

    • Author(s)
      De Looze, C. and Scherer, S. and Vaughan, B. and Campbell
    • Journal Title

      Speech Communication

      Volume: 58 Pages: 11-34

    • DOI

      10.1016/j.specom.2013.10.002

    • Related Report
      2014 Annual Research Report 2013 Research-status Report
    • Peer Reviewed
  • [Journal Article] "Classification of social laughter in natural conversational speech",2013

    • Author(s)
      Tanaka Hiroki and Nick Campbell
    • Journal Title

      Journal of Computer Speech & Language

      Volume: Vol 28 (1) Issue: 1 Pages: 314-325

    • DOI

      10.1016/j.csl.2013.07.004

    • Related Report
      2013 Research-status Report
  • [Journal Article] Refereed Original Articles D64: A Corpus of Richly Recorded Conversational Interaction2013

    • Author(s)
      Catharine Oertel, Fred Cummins, Jens Edlund, Petra Wagner, Nick Campbell
    • Journal Title

      Journal on Multimodal User Interfaces

      Volume: Volume 7, issue 1-2 Issue: 1-2 Pages: 19-28

    • DOI

      10.1007/s12193-012-0108-6

    • Related Report
      2013 Research-status Report
  • [Journal Article] D64: a corpus of richly recorded conversational interaction2013

    • Author(s)
      Catharine Oertel, Fred Cummins, Jens Edlund, Petra Wagner, Nick Campbell
    • Journal Title

      Multimodal User Interfaces

      Volume: Volume 7

    • Related Report
      2012 Research-status Report
  • [Journal Article] Designing and Implementing a Platform for Collecting Multi-Modal Data of Human-Robot Interaction.2012

    • Author(s)
      Vaughan, B., Han, G. J., Gilmartin, E & Campbell, N
    • Journal Title

      Acta Polytechnica Hungarica

      Volume: 9

    • Related Report
      2012 Research-status Report
  • [Journal Article] Analysis of acoustic-prosodic features of spontaneous expressive speech2012

    • Author(s)
      Carlos Toshinori Ishi, Nick Campbell,
    • Journal Title

      Revista de Estudos da Linguagem

      Volume: 12

    • Related Report
      2012 Research-status Report
  • [Presentation] Voice Quality: not only about "you" but also about "your interlocutor"2015

    • Author(s)
      Ya Li, Nick Campbell, JianHua Tao
    • Organizer
      ICASSP
    • Place of Presentation
      Brisbane, Australia
    • Year and Date
      2015-04-19 – 2015-04-24
    • Related Report
      2014 Annual Research Report
  • [Presentation] Social sequence analysis: temporal sequences in interactional conversations2014

    • Author(s)
      Francesca Bonin, Carl Vogel and Nick Campbell
    • Organizer
      5th IEEE Conference on Cognitive Infocommunications(CogInfoCom)
    • Place of Presentation
      Vietri sul Mare, Italy
    • Year and Date
      2014-11-05 – 2014-11-07
    • Related Report
      2014 Annual Research Report
  • [Presentation] An introduction to the TCD D_ANS Corpus - a multimodal multimedia biometric corpus of spoken social 2014,interaction2014

    • Author(s)
      Nick Campbell
    • Organizer
      Interspeech
    • Place of Presentation
      MA3HMI,Singapore
    • Year and Date
      2014-09-14
    • Related Report
      2014 Annual Research Report
    • Invited
  • [Presentation] Beyond D64 - A fresh look at multimodal dialogue systems from the FastNet point of view - how we are incorporating different modalities into an interactive speech synthesis system., Multimodality- interdisciplinary perspectives: new questions and new methodologies2014

    • Author(s)
      Nick Campbell
    • Organizer
      INALCO
    • Place of Presentation
      Universite Paris 8 et CNRS, France
    • Year and Date
      2014-06-06
    • Related Report
      2014 Annual Research Report
    • Invited
  • [Presentation] The D-ANS corpus: the Dublin-Autonomous Nervous System corpus of 2014 biosignal and multimodal recordings of conversational speech2014

    • Author(s)
      Shannon Hennig, Ryad Chellali and Nick Campbell
    • Organizer
      The 9th edition of the Language Resources and Evaluation Conference, ELRA
    • Place of Presentation
      Reykjavik, Iceland
    • Year and Date
      2014-05-26 – 2014-05-31
    • Related Report
      2014 Annual Research Report
  • [Presentation] "Laugher and Topic Transition in Multiparty Conversation"2013

    • Author(s)
      Boeck, Bonin, Campbell, Edlund, De Kok, Poppe, Traum
    • Organizer
      Proceedings of the SIGDIAL 2013 Conference
    • Place of Presentation
      Metz, France
    • Related Report
      2013 Research-status Report
  • [Presentation] "Investigating fine temporal dynamics of prosodic and lexical accommodation"2013

    • Author(s)
      Francesca Bonin, Celine De Looze, Sucheta Ghosh, Emer Gilmartin, Carl Vogel, Anna Polychroniou, Hugues Salamin, Alessandro Vinciarelli and Nick Campbell
    • Organizer
      14th Annual Conference of the International Speech Communication Association, INTERSPEECH, 2013
    • Place of Presentation
      Lyon, France
    • Related Report
      2013 Research-status Report
  • [Presentation] "Exploring the Role of Laughter in Multiparty Conversation"2013

    • Author(s)
      Emer Gilmartin, Francesca Bonin, Carl Vogel and Nick Campbell
    • Organizer
      Proceedings (SemDial), semdial 2013,DialDam
    • Place of Presentation
      Amsterdam, Neatherlands.
    • Related Report
      2013 Research-status Report
  • [Presentation] Herme, Yet Another Interactive Conversational Robot2013

    • Author(s)
      JinGuang Han, Gilmartin, E. , Campbell, N.
    • Organizer
      Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on
    • Place of Presentation
      Geneva, Switzerland.
    • Related Report
      2013 Research-status Report
  • [Presentation] Identification of cues for the automatic detection of hotspots in natural conversation

    • Author(s)
      Oertel, C., De Looze, C., Campbell, N and Wagner, P
    • Organizer
      SSPnet-COST 2012 PINK International Conference
    • Place of Presentation
      Budapest, Hungary
    • Related Report
      2012 Research-status Report
  • [Presentation] The Herme Database of Spontaneous Multimodal Human-‐Robot Dialogues

    • Author(s)
      Han, J., Gilmartin, E., De Looze, C., Vaughan, B., & Nick Campbell,
    • Organizer
      Conference on Language Resources and Evaluation (LREC'12),
    • Place of Presentation
      Istanbul, Turkey
    • Related Report
      2012 Research-status Report
  • [Presentation] An audiovisual political speech analysis incorporating eye-tracking and perception data

    • Author(s)
      Stefan Scherer, Georg Layher, John Kane, Heiko Neumann and Nick Campbell
    • Organizer
      Conference on Language Resources and Evaluation (LREC'12)
    • Place of Presentation
      Istanbul, Turkey
    • Related Report
      2012 Research-status Report
  • [Presentation] Laughter and Topic Changes: Temporal Distribution and Information Flow

    • Author(s)
      Bonin, Francesca and Nick Campbell and Carl Vogel
    • Organizer
      3rd IEEE Conference on Cognitive Infocommunications
    • Place of Presentation
      Kosice, Slovakia
    • Related Report
      2012 Research-status Report
  • [Presentation] Vers une mesure automatique de l’adaptation prosodique en interaction conversationnelle

    • Author(s)
      De Looze, C., Scherer, S., Vaughan, B. & Campbell, N.
    • Organizer
      29eme Journees d’Etude sur la Parole (JEP 2012)
    • Place of Presentation
      Grenoble, France
    • Related Report
      2012 Research-status Report
  • [Presentation] More than just words: Building a chatty robot

    • Author(s)
      Gilmartin, E., Campbell, N
    • Organizer
      International Workshop on Spoken Dialog Systems (IWSDS 2012)
    • Place of Presentation
      Paris,France
    • Related Report
      2012 Research-status Report
  • [Presentation] Temporal distribution of laughter in conversation

    • Author(s)
      Francesca Bonin, Nick Campbell, Carl Vogel
    • Organizer
      Third Interdisciplinary Workshop on Laughter and other Non‐Verbal Vocalisations in Speech
    • Place of Presentation
      Dublin, Ireland
    • Related Report
      2012 Research-status Report
  • [Presentation] Investigating the use of Non-verbal Cues in Human-Robot Interaction with a Nao robot

    • Author(s)
      JingGuang Han, Nick Campbell, Kristiina Jokinen, Graham Wilcock
    • Organizer
      IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom)
    • Place of Presentation
      Kosice, Slovakia
    • Related Report
      2012 Research-status Report
  • [Presentation] Non-verbal cognitive skills and autistic conditions: An analysis and training tool

    • Author(s)
      Hiroki Tanaka, Sakriani Sakti, Graham Neubig, Tomoki Toda, Nick Campbell, Satoshi Nakamura
    • Organizer
      IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom)
    • Place of Presentation
      Kosice, Slovakia
    • Related Report
      2012 Research-status Report
  • [Presentation] How Do We React to Context? Annotation of Individual and Group Engagement in a Video Corpus, Privacy, Security, Risk and Trust (PASSAT)

    • Author(s)
      Francesca Bonin, Ronald Bock, Nick Campbell
    • Organizer
      International Conference on and 2012 International Confernece on Social Computing (SocialCom)
    • Place of Presentation
      Amsterdam, Netherland
    • Related Report
      2012 Research-status Report
  • [Presentation] Priming, Timing, and the Phatic Component in Machine-Mediated Dialogue

    • Author(s)
      Emer Gilmartin, Celine De Looze, Nick Campbell
    • Organizer
      The Listening Talker (LISTA 2012)
    • Place of Presentation
      Edinburgh, Scotland
    • Related Report
      2012 Research-status Report
  • [Presentation] Social aspects and speechability in CogInfoCom systems

    • Author(s)
      N. Campbell
    • Organizer
      CogInfoCom 2012
    • Place of Presentation
      Kosice, Slovakia
    • Related Report
      2012 Research-status Report
    • Invited
  • [Book] Annotating the TCD D-ANS Corpus - a multimodal multimedia monolingual biometric corpus of spoken social interaction2015

    • Author(s)
      Nick Campbell and Shannon Hennig
    • Publisher
      Springer Verlag
    • Related Report
      2014 Annual Research Report
  • [Book] SpeechProsody in Speech Synthesis: Modeling and generation of prosody for high quality and flexible speech synthesis(Expressivity in Interactive Speech Synthesis; Some Paralinguistic and Nonlinguistic Issues of Speech Prosody for 2015 Conversational Dialogue Systems )2015

    • Author(s)
      Nick Campbell and Ya Li
    • Publisher
      Springer
    • Related Report
      2014 Annual Research Report
  • [Book] Coverbal Synchrony in Human-Machine Interaction2014

    • Author(s)
      Matej Rojc, Niick Campbell
    • Publisher
      Boca Raton, CRC Press
    • Related Report
      2014 Annual Research Report
  • [Book] Natural Interaction with Robots, Knowbots and Smartphones (More Than Just Words: Building a Chatty Robot)2014

    • Author(s)
      Emer Gilmartin, Nick Campbell
    • Publisher
      Springer
    • Related Report
      2014 Annual Research Report
  • [Book] More Than Just Words: Building a Chatty Robot / Natural Interaction with Robots, Knowbots and Smartphones2014

    • Author(s)
      Emer Gilmartin, Nick Campbell
    • Publisher
      Springer
    • Related Report
      2013 Research-status Report
  • [Book] Speech Technology and Conversational Activity in Human-Machine Interaction2013

    • Author(s)
      Matej Rojc & Nick Campbell
    • Total Pages
      434
    • Publisher
      CRC Press
    • Related Report
      2013 Research-status Report
  • [Book] Speech, Gaze, anmd Affect: concepts of reactive and natural human-machine interaction techniques employing ECAs with personality2013

    • Author(s)
      Matej Rojc & Nick Capbell (eds)
    • Publisher
      Science Publishers, Jersey
    • Related Report
      2012 Research-status Report
  • [Book] Rhythm, Melody, and Harmony in Speech2012

    • Author(s)
      D. Gibbon, D. Hirst, and N. Campbell
    • Publisher
      Polish Phonetic Association, Poznan
    • Related Report
      2012 Research-status Report
  • [Book] Conversational Involvement and Synchronous Nonverbal Behaviour in, Esposito,"Cognitive Behavioural Systems"2012

    • Author(s)
      A., Vinciarelli, A., Hoffmann, R., V. (Eds.) [Altmann, U., Oertel, C., & Campbell, N.]
    • Publisher
      Springer
    • Related Report
      2012 Research-status Report

URL: 

Published: 2013-05-31   Modified: 2021-04-07  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi