• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development and Analysis of Multimodal Corpora of English Public Speaking Performance: Interdisciplinary Project of Linguistics and Engineering

Research Project

Project/Area Number 15K12416
Research Category

Grant-in-Aid for Challenging Exploratory Research

Allocation TypeMulti-year Fund
Research Field Educational technology
Research InstitutionKyushu University

Principal Investigator

Fuyuno Miharu  九州大学, 芸術工学研究院, 助教 (30642681)

Co-Investigator(Kenkyū-buntansha) 山下 友子  芝浦工業大学, 工学部, 助教 (10726334)
中島 祥好  九州大学, 芸術工学研究院, 教授 (90127267)
齊藤 剛史  九州工業大学, 大学院情報工学研究院, 准教授 (10379654)
Project Period (FY) 2015-04-01 – 2018-03-31
Project Status Completed (Fiscal Year 2017)
Budget Amount *help
¥3,770,000 (Direct Cost: ¥2,900,000、Indirect Cost: ¥870,000)
Fiscal Year 2017: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Fiscal Year 2016: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2015: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Keywordsマルチモーダルコーパス / パブリックスピーキング / 英語教育 / モーショントラッキング / スピーチ / プレゼンテーション / 音声解析 / アイコンタクト / コーパス / マルチモーダル / 行動分析 / コーパス言語学 / 動作分析 / スピーチ教育 / 英語学習
Outline of Final Research Achievements

This research aimed at developing multimodal corpora of English public speaking performance by both native/non-native speakers of English. We collected multichannnel input data of Japanese English learners and NSEs. The data mainly included high resolution video data, sound data, script data, and performance evaluation data. The data were analyzed using 2D motion tracking method and sound analysis software. From the analysis, the characteristics of eye-contact movement and phonological elements of highly evaluated speakers were revealed. The result was then applied into developing a teaching manual for effective public speaking.

Report

(4 results)
  • 2017 Annual Research Report   Final Research Report ( PDF )
  • 2016 Research-status Report
  • 2015 Research-status Report
  • Research Products

    (23 results)

All 2018 2017 2016 2015 Other

All Int'l Joint Research (1 results) Journal Article (9 results) (of which Peer Reviewed: 7 results,  Open Access: 1 results,  Acknowledgement Compliant: 5 results) Presentation (11 results) (of which Int'l Joint Research: 9 results) Remarks (1 results) Patent(Industrial Property Rights) (1 results)

  • [Int'l Joint Research] National University of Ireland(Ireland)

    • Related Report
      2015 Research-status Report
  • [Journal Article] Multimodal analysis of public speaking performance by EFL learners: Applying deep learning to understanding how successful speakers use facial movement2018

    • Author(s)
      Fuyuno, M., Komiya, R., and Saitoh, T.
    • Journal Title

      The Asian Journal of Applied Linguistics

      Volume: 4 Pages: 117-129

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] The effect of virtual eyes on psychological nervousness of public speakers2017

    • Author(s)
      Fuyuno, M., Yamashita, Y., Nakajima, Y.
    • Journal Title

      Proceedings of the 33rd Annual Meeting of the International Society for Psychophysics

      Volume: 33 Pages: 128-131

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 英語パブリックスピーキングにおける音声ポーズパターンとアイコンタクト動作パターンがパフォーマンスの総合評価へ及ぼす影響に関する統計的考察2017

    • Author(s)
      冬野美晴, 山下友子
    • Journal Title

      マルチモーダルコーパスデータに基づくパブリックスピーチの統計的解析

      Volume: なし Pages: 1-15

    • Related Report
      2016 Research-status Report
    • Acknowledgement Compliant
  • [Journal Article] スピーチ指導のための映像情報を用いた話者の顔解析2017

    • Author(s)
      小宮凛子, ワラポン・チンサティ, 齊藤剛史, 冬野美晴
    • Journal Title

      DIA2017

      Volume: なし Pages: 195-200

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Head pose estimation and motion analysis of public speaking videos2017

    • Author(s)
      Komiya, R., Saitoh, T., Fuyuno, M., Yamashita, Y., Nakajima, Y.
    • Journal Title

      International Journal of Software Innovation

      Volume: 5 Pages: 57-71

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Head pose estimation and movement analysis for speech scene2017

    • Author(s)
      Komiya, R., Saitoh, T., Fuyuno, M., Yamashita, Y., Nakajima, Y.
    • Journal Title

      Proc. of 15th IEEE/ACIS International Conference on Computer and Information Science

      Volume: なし Pages: 409-413

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Three Aspects of Public Speeches with Competent Speakers: Speech Pauses, Head Movements, and Lexical Choices2015

    • Author(s)
      Yamashita, Y. & Fuyuno, M.
    • Journal Title

      STEM Journal

      Volume: 5 Pages: 215-231

    • Related Report
      2015 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] アメリカの大学の卒業式における著名人による祝辞スピーチの録画映像の分析:音声の無音区間と顔上げ動作2015

    • Author(s)
      山下友子・冬野美晴
    • Journal Title

      ATEM Journal

      Volume: 20 Pages: 33-44

    • NAID

      130007817494

    • Related Report
      2015 Research-status Report
    • Peer Reviewed
  • [Journal Article] アジア圏グローバルビジネスパーソンを対象とした英語スキルのニーズ分析:グローバル人材育成教育に向けた考察2015

    • Author(s)
      冬野美晴
    • Journal Title

      Studies in English Teaching and Learning in East Asia

      Volume: 5 Pages: 13-27

    • Related Report
      2015 Research-status Report
  • [Presentation] 没入型バーチャル・リアリティによる英語パブリックスピーキング練習用システムの開発:マルチモーダルコーパス分析に基づく教材の展開2017

    • Author(s)
      冬野美晴
    • Organizer
      外国語教育メディア学会全国大会
    • Related Report
      2017 Annual Research Report
  • [Presentation] Analyzing Public Speaking for EAP Pedagogy: Factors of Better Performance2017

    • Author(s)
      Fuyuno, M., Komiya, R., Saitoh, T
    • Organizer
      Faces of English 2: Teaching and Researching Academic and Professional English
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Quantitative analysis of English public speaking presented by Japanese EFL learners2017

    • Author(s)
      Liu, S., Nakajima, Y., Yamashita, Y., Fuyuno, M.
    • Organizer
      the 6th Conference of the Asia-Pacific Society for the Cognitive Sciences of Music
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] スピーチ指導のための顔特徴点を用いた話者頭部の動き解析2016

    • Author(s)
      小宮凜子, 齊藤剛史, 冬野 美晴, 山下 友子, 中島 祥好
    • Organizer
      電子情報通信学会総合大会
    • Place of Presentation
      九州大学
    • Year and Date
      2016-03-16
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] Semantic Structure, Speech Units and Facial Movements: Multimodal Corpus Analysis of English Public Speakin2016

    • Author(s)
      Fuyuno, M., Yamashita, Y., Saitoh, T., & Nakajima, Y.
    • Organizer
      8th International Conference on Corpus Linguist
    • Place of Presentation
      university of Malaga
    • Year and Date
      2016-03-02
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] Developing effective instructions to decrease Japanese speaker' s nervousness during English and Japanese public speeches: Evidence from psychological and physiological measurements2016

    • Author(s)
      Fuyuno, M., Yamada, Y., Yamashita, Y., Yoshitaka Nakajima
    • Organizer
      31st International Congress of Psychology 2016 (ICP2016)
    • Place of Presentation
      パシフィコ横浜
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Multimodal Corpus Approach to Speech Psychology: The Effect of Public Speakers’ Facial Movements on Audience Reception2016

    • Author(s)
      Fuyuno, M., Yamashita, Y., Saitoh, T., Nakajima, Y.
    • Organizer
      31st International Congress of Psychology 2016 (ICP2016)
    • Place of Presentation
      パシフィコ横浜
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] 日本人大学生の英語プレゼンテーションにおける効果的な間の取り方2016

    • Author(s)
      山下友子, 冬野美晴, 横森大輔
    • Organizer
      日本認知科学会研究分科会「間合い―時空間インタラクション」
    • Place of Presentation
      東京工業大学
    • Related Report
      2016 Research-status Report
  • [Presentation] Analysis and application of multimodal corpora of English public speaking by EFL learners: Toward evidence-based teaching2015

    • Author(s)
      Miharu Fuyuno
    • Organizer
      International Conference on Corpus Linguistics and Technology Advancement 2015
    • Place of Presentation
      Hong Kong Institution of Education
    • Year and Date
      2015-12-17
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] マルチモーダルコーパスを用いた英語母語話者によるパブリックスピーキング・パフォーマンスの分析:発話速度・ジェスチャー・語彙特性の観点から2015

    • Author(s)
      冬野 美晴, 山下 友子
    • Organizer
      英語コーパス学会第41回全国研究大会
    • Place of Presentation
      愛知大学
    • Year and Date
      2015-10-04
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] An analysis of speech pauses, head movements, and lexical choices in graduation speeches at the universities in the United States2015

    • Author(s)
      Yamashita, Y. & Fuyuno, M.
    • Organizer
      The 19th STEM International Conference
    • Place of Presentation
      Nazare University
    • Year and Date
      2015-05-16
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Remarks] 研究室ウェブサイト

    • URL

      http://global-human-resource.net/

    • Related Report
      2015 Research-status Report
  • [Patent(Industrial Property Rights)] パブリックスピーキング支援装置、及びプログラム2017

    • Inventor(s)
      冬野美晴、山下友子、齊藤剛史、中島祥好
    • Industrial Property Rights Holder
      冬野美晴、山下友子、齊藤剛史、中島祥好
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      2017-077706
    • Filing Date
      2017
    • Related Report
      2017 Annual Research Report

URL: 

Published: 2015-04-16   Modified: 2019-03-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi