Facilitaiton effect on speech production by gesture and its neural mechanism

• Project/Area Number: 15K21066
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Multi-year Fund
• Research Field: Experimental psychology; Cognitive science
• Research Institution: Kyushu University
• Principal Investigator: Oi Misato 九州大学, 附属図書館, 准教授 (70579763)
• Project Period (FY): 2015-04-01 – 2019-03-31
• Project Status: Completed (Fiscal Year 2018)
• Budget Amount: ¥3,900,000 (Direct Cost: ¥3,000,000、Indirect Cost: ¥900,000); Fiscal Year 2016: ¥2,080,000 (Direct Cost: ¥1,600,000、Indirect Cost: ¥480,000); Fiscal Year 2015: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
• Keywords: ジェスチャー / 発話 / 脳活動 / 第二言語 / 神経機構 / 二言語併用 / 促進効果 / 二言語併用者 / 促進機能

Outline of Final Research Achievements

This study aimed to investigate whether gesture production facilitates speech production by affecting lexical retrieval and/or organizing information for speech with examining effects of factors of individual differences of frequency of gesture production and spontaneity of gesture production.Furthermore, this study investigated the neural bases of co-speech gesture production that had been not actively investigated by previous studies.

Academic Significance and Societal Importance of the Research Achievements

本研究の成果は，ジェスチャーが発話で果たす促進的な機能の詳細とその神経機構，及び，これらとジェスチャー産出頻度の個人差と自発性との関連の解明である。これらの知見の学問的な意義は，言語とジェスチャーの関連性とその神経基盤，言語の起源とジェスチャーの関係を解明する端緒となることにある。また，社会的な意義は，第二言語の学習，ひいては，学習全般にジェスチャーを取り入れる効果を，行動データと生理データの2側面から明らかにする点にある。

Research Products

• [Journal Article] Exploring the Relationships between Reading Behavior Patterns and Learning Outcomes based on Log Data from e-books: A Human Factor Approach (2018)
  • Author(s): Yin, C.,Yamada, M., Oi,M., Shimada, A.,Okubo, F.,Kojima, K.,Ogata,H.
  • Journal Title: International Journal of Human-Computer Interaction Volume: 35 Issue: 4-5 Pages: 313-322
  • DOI: 10.1080/10447318.2018.1543077
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] The neural correlates of biomechanical constraints in hand laterality judgment task performed from other person’s perspective: A near-infrared spectroscopy study (2017)
  • Author(s): Meng Shuang、Oi Misato、Saito Godai、Saito Hirofumi
  • Journal Title: PLOS ONE Volume: 12 Issue: 9 Pages: e0183818-e0183818
  • DOI: 10.1371/journal.pone.0183818
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Learning analytics of the relationships among self-regulated learning, learning behaviors, and learning performance (2017)
  • Author(s): Yamada, M., Shimada, A., Okubo, F., Oi, M., Kojima, K. & Ogata, H.
  • Journal Title: Research and Practice in Technology Enhanced Learning Volume: 12 Issue: 1 Pages: 1-17
  • DOI: 10.1186/s41039-017-0053-9
  • Peer Reviewed
• [Journal Article] Cognitive empathy modulates the visual perception of human-like body postures without imitation. (2016)
  • Author(s): Oi, M., Ito, H., Saito, H., Meng, S., & Palacios, V. A.
  • Journal Title: Journal of Cognitive Psychology Volume: 28 Issue: 3 Pages: 319-328
  • DOI: 10.1080/20445911.2015.1127250
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Obstruction increases activation in the right inferior frontal gyrus (2016)
  • Author(s): Liu, T., Saito, H., & Oi, M.
  • Journal Title: Social Neuroscience Volume: 11 Issue: 4 Pages: 344-352
  • DOI: 10.1080/17470919.2015.1088469
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] The neural mechanism of biomechanical constraints in the hand laterality judgment task: A near-infrared spectroscopy study. (2016)
  • Author(s): Meng, S., Oi, M., Sekiyama, K., & Saito, H.
  • Journal Title: Neuroscience Letters Volume: 627 Pages: 211-215
  • DOI: 10.1016/j.neulet.2016.05.069
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Understand after like, viewer's delight? A fNIRS study of order-effect in combined hedonic and cognitive appraisal of art. (2016)
  • Author(s): Pelowski, M., Oi, M., Liu, T., Meng, S., Saito, G., & Saito, H.
  • Journal Title: Acta Psychologica Volume: 170 Pages: 127-138
  • DOI: 10.1016/j.actpsy.2016.06.005
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Role of the right inferior frontal gyrus in turn-based cooperation and competition: A near-infrared spectroscopy study. (2015)
  • Author(s): Liu, T., Saito, H., & Oi, M.
  • Journal Title: Brain and Cognition Volume: 99 Pages: 17-23
  • DOI: 10.1016/j.bandc.2015.07.001
  • Peer Reviewed
• [Presentation] 高齢者向け学習支援システムのためのデュアルタブレット・ユーザインタフェースの提案と実現 (2019)
  • Author(s): Lu Min・田村かおり・岡本剛・大井京・島田敬士・畑埜晃平・山田政寛・木實新一
  • Organizer: 情報処理学会 第81回全国大会
• [Presentation] 自学学習中における脳波・視線の同時計測・分析システムの開発 (2019)
  • Author(s): 田村かおり・岡本剛・大井京・島田敬士・畑埜晃平・山田政寛・Lu Min・木實新一
  • Organizer: 情報処理学会 第81回全国大会
• [Presentation] 脳波・視線同時計測による「学びのつまずき」推定システムの開発 (2019)
  • Author(s): 田村かおり・岡本剛・大井京・島田敬士・畑埜晃平・山田政寛・陸忞・木實新一
  • Organizer: 日本生体医工学会九州支部学術講演会
• [Presentation] Extending Learning Analytics Platforms to Support Elderly People (2018)
  • Author(s): Konomi, S., Hatano, K., Inaba, M., Oi, M., Okamoto, T., Okubo, F., Shimada, A., Wang, J., Yamada, M., & Yamada, Y.
  • Organizer: The 12th International Workshop on Information Search, Integration, and Personalization (ISIP2018)
  • Int'l Joint Research
• [Presentation] BR-Map: Concept Map System Using e-Book logs (2018)
  • Author(s): Yamada, M., Shimada, A., Oi, M., Taniguchi, Y., and Konomi, S.
  • Organizer: Proceedings of CELDA 2018
  • Int'l Joint Research
• [Presentation] Rlationships between collaborative problem solving, learning performance and learrning behavior in science education (2018)
  • Author(s): Chen, L., Uemura, H., Hao, H., Goda, Y., Okubo, F., Taniguchi, Y., Oi, M., Konomi, S., Ogata, H., Yamada, M.
  • Organizer: Proceedings of IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)
  • Int'l Joint Research
• [Presentation] Instructional Design and Evaluation of Science Education to Improve Collaborative Problem Solving Skills. (2018)
  • Author(s): Chen, L., Uemura, H., Goda, Y., Okubo, F., Taniguchi, Y., Oi, M., Konomi, S., Ogata, H. & Yamada, M.
  • Organizer: Society for Information Technology & Teacher Education International Conference (SITE2018)
  • Int'l Joint Research
• [Presentation] Effects of prior knowledge of high achievers on use of e-book highlights and annotations (2017)
  • Author(s): Oi, M., Okubo, F., Taniguchi, Y., Yamada, M., & Konomi, S.
  • Organizer: 25th International Conference on Computers in Education
  • Int'l Joint Research
• [Presentation] Are learning logs related to procrastination? from the viewpoint of self-regulated learning (2017)
  • Author(s): Yamada, M., Oi, M., & Konomi, S.
  • Organizer: 14th International Conference on Cognition and Exploratory Learning in Digital Age
  • Int'l Joint Research
• [Presentation] Reproducibility of findings from educational big data: A preliminary study (2017)
  • Author(s): Oi, M., Yamada, M., Okubo, F., Shimada, A., & Ogata, H.
  • Organizer: 7th International Learning Analytics and Knowledge (LAK) Conference
  • Place of Presentation: Vancouver, Canada
  • Int'l Joint Research
• [Presentation] Finding traces of high and low achievers by analyzing undergraduates’ e-book logs (2017)
  • Author(s): Oi, M., Yamada, M., Okubo, F., Shimada, A., & Ogata, H
  • Organizer: Cross-LAK2017
  • Place of Presentation: Vancouver, Canada
  • Int'l Joint Research
• [Presentation] M2B System: A digital learning platform for traditional classrooms in university (2017)
  • Author(s): Ogata, H., Taniguchi, Y., Suehiro, D., Shimada, A., Oi, M., Okubo, F., Yamada, M., & Kojima, K.
  • Organizer: 7th International Learning Analytics and Knowledge (LAK) Conference
  • Place of Presentation: Vancouver, Canada
  • Int'l Joint Research
• [Presentation] Learning analytics in ubiquitous learning environments: Self-regulated learning perspective (2016)
  • Author(s): Yamada, M., Okubo, F., Oi, M., Shimada, A., Kojima, K., & Ogata, H.
  • Organizer: 24th International Conference on Computers in Education
  • Place of Presentation: Mumbai, India
  • Int'l Joint Research
• [Presentation] Learning Activity Features of High Performance Students. (2016)
  • Author(s): Okubo, F., Hirokawa, S., Oi, M., Yin, C., Shimada, A., Kojima, K., Yamada, M., & Ogata, H.
  • Organizer: Cross-LAK2016
  • Place of Presentation: Edinburgh, United Kingdom
  • Int'l Joint Research
• [Presentation] Brain-to-brain coupling of neural correlates of action and speech memory based deception and its detection: A Near-Infrared Spectroscopy (NIRS) study (2016)
  • Author(s): Saito, G., Gyoba, J., Yamada, R., Itsukushima, Y., Oi, M., Lin, C., & Saito, H.
  • Organizer: 31st International Congress of Psychology
  • Place of Presentation: Yokohama, Japan
  • Int'l Joint Research
• [Presentation] e-Book-based Learning Analytics in University Education (2015)
  • Author(s): Ogata, H., Yin C., Oi, M., Okubo, F., Shimada, A., Kojima, K. and Yamada, M.
  • Organizer: 23rd International Conference on Computers in Education
  • Place of Presentation: Hangzhou, China
  • Year and Date: 2015-11-30
  • Int'l Joint Research
• [Presentation] Analysis of preview and review patterns in undergraduates’ e-book logs. (2015)
  • Author(s): Oi, M., Okubo, F., Shimada, A., Yin, C., & Ogata, H.
  • Organizer: 23rd International Conference on Computers in Education
  • Place of Presentation: Hangzhou, China
  • Year and Date: 2015-11-30
  • Int'l Joint Research
• [Presentation] Analysis of Links among E-books in Undergraduates’ E-Book Logs (2015)
  • Author(s): Oi, M., Yin, C., Okubo, F., Shimada, A., Kojima, K., Yamada, M., & Ogata, H
  • Organizer: 23rd International Conference on Computers in Education
  • Place of Presentation: Hangzhou, China
  • Year and Date: 2015-11-30
  • Int'l Joint Research
• [Presentation] Yin, C., Okubo, F., Shimada, A., Hirokawa, S., Ogata H., & Oi, M.Identifying and analyzing the learning behaviors of students using e-books. (2015)
  • Author(s): Yin, C., Okubo, F., Shimada, A., Hirokawa, S., Ogata H., & Oi, M.
  • Organizer: 23rd International Conference on Computers in Education
  • Place of Presentation: Hangzhou, China
  • Year and Date: 2015-11-30
  • Int'l Joint Research
• [Presentation] Visualization of e-book learning logs (2015)
  • Author(s): Hirokawa, S., Yin, C., Wang, J., Oi, M., & Ogata, H.
  • Organizer: 23rd International Conference on Computers in Education
  • Place of Presentation: Hangzhou, China
  • Year and Date: 2015-11-30
  • Int'l Joint Research
• [Presentation] Analysis of Preview Behavior in E-Book System (2015)
  • Author(s): Shimada, A., Okubo, F., Yin, C., Oi, M., Kojima, K., Yamada, M., & Ogata, H.
  • Organizer: 23rd International Conference on Computers in Education
  • Place of Presentation: Hangzhou, China
  • Year and Date: 2015-11-30
  • Int'l Joint Research
• [Presentation] Analyzing the Features of Learning Behaviors of Students using e-Books (2015)
  • Author(s): Yin, C., Okubo, F., Shimada, A., Oi, M., Hirokawa, Yamada, M., Kojima, K., & Ogata H.
  • Organizer: 23rd International Conference on Computers in Education
  • Place of Presentation: Hangzhou, China
  • Year and Date: 2015-11-30
  • Int'l Joint Research
• [Book] Smart Sensors at the IoT Frontier (2017)
  • Author(s): Yasuura, H., Kyung, C.-M., Liu, Y., Lin, Y.-L. (Eds.)
  • Total Pages: 378
  • Publisher: Springer International Publishing
  • ISBN: 9783319553450