• 研究課題をさがす
  • 研究者をさがす
  • KAKENの使い方
  1. 前のページに戻る

Theory, Methodology and Tools for Tailoring CAPT Feedback

研究課題

研究課題/領域番号 23K00679
研究種目

基盤研究(C)

配分区分基金
応募区分一般
審査区分 小区分02100:外国語教育関連
研究機関会津大学

研究代表者

Pyshkin Evgeny  会津大学, コンピュータ理工学部, 上級准教授 (50794088)

研究分担者 Mozgovoy Maxim  会津大学, コンピュータ理工学部, 上級准教授 (60571776)
BLAKE John  会津大学, コンピュータ理工学部, 上級准教授 (80635954)
研究期間 (年度) 2023-04-01 – 2026-03-31
研究課題ステータス 交付 (2023年度)
配分額 *注記
4,290千円 (直接経費: 3,300千円、間接経費: 990千円)
2025年度: 1,040千円 (直接経費: 800千円、間接経費: 240千円)
2024年度: 1,560千円 (直接経費: 1,200千円、間接経費: 360千円)
2023年度: 1,690千円 (直接経費: 1,300千円、間接経費: 390千円)
キーワードCAPT feedback / personalization / targeting / multimodality / multilingual CAPT / intelligent CAPT / multimodal / intonation / pitch evaluation / speech visualization
研究開始時の研究の概要

This project consists of three core work packages: Formulation of theoretical grounds for using dynamic system models in CAPT systems; Enhancing the CAPT environment with attitudinal and contrastive pronunciation exercises; Refining mobile components UI with respect to evaluation of speech dynamics.

研究実績の概要

The focus of works in AY2023 was on developing models, interfaces, and software to improve CAPT feedback with respect to supporting contrastive pronunciation in context and investigating the possibilities of accent recognition for personalizing the CAPT exercises and feedback addressing the learners' L1 background.
Studies on multimodal modeling of mora-timed Japanese enhanced our CAPT environment towards its multilingual setup. We published an extensive study of the important aspects of multilingual multimodality and the specific language features affecting the architecture and tools of an intelligent CAPT environment. By selecting one language from each of three different language families, we demonstrated how a single environment may be tailored to cater for
different target languages. We demonstrated that since the applied underlying mathematical and phonological models, as well as the feedback production algorithms, are based on sound signal processing and modeling rather than on particular languages, the system is language-agnostic and serves as an open toolkit for developing phrasal intonation training exercises for an open selection of languages. We succeeded to describe our response to the challenges in visualizing and segmenting recorded pitch signals and modeling the language melody and rhythm necessary for such a multilingual adaptation, particularly for tonal syllable-timed and mora-timed languages.
We achieved the interesting cross-disciplinary results on connecting the CAPT feedback models to other learners' backgrounds including music, languages, and arts.

現在までの達成度 (区分)
現在までの達成度 (区分)

2: おおむね順調に進展している

理由

The experiments on accent classification using Speech Accent Archive and CNN enabled the possibilities to define a model for CAPT exercise targeting depending on learners' L1. Empirical studies and assessment of suggest feedback tailoring models and interfaces created good grounds for contribution to the theory of dynamic assessment and mediation for moving learners beyond the plateau (to be published in AY2024).

今後の研究の推進方策

We need further efforts to integrate the features assessed in scope of research versions of learning software into the intelligent CAPT environment. Assessment of existing tool in the classroom and by individual learners will help to better understand what are the most productive models for user feedback personalization. Also, we will describe our findings on how to contextualize the speech situations using a variety of CAPT feedback production forms and what is the role of iCAPT systems in enabling learners to move beyond their learning plateau. We will rely on the concepts of the sociocultural theory, such as mediation through the zone of proximal development, thus connecting the pedagogical, linguistic, and computational aspects of CAPT design and development.

報告書

(1件)
  • 2023 実施状況報告書
  • 研究成果

    (11件)

すべて 2024 2023 その他

すべて 国際共同研究 (2件) 雑誌論文 (3件) (うち国際共著 2件、 査読あり 2件、 オープンアクセス 1件) 学会発表 (5件) (うち国際学会 4件、 招待講演 1件) 学会・シンポジウム開催 (1件)

  • [国際共同研究] St. Petersburg Polytechnic University(ロシア連邦)

    • 関連する報告書
      2023 実施状況報告書
  • [国際共同研究] Hachinohe College(日本)

    • 関連する報告書
      2023 実施状況報告書
  • [雑誌論文] An Open CAPT System for Prosody Practice: Practical Steps towards Multilingual Setup2024

    • 著者名/発表者名
      Blake John、Bogach Natalia、Kusakari Akemi、Lezhenin Iurii、Khaustova Veronica、Xuan Son Luu、Nguyen Van Nhi、Pham Nam Ba、Svechnikov Roman、Ostapchuk Andrey、Efimov Dmitrei、Pyshkin Evgeny
    • 雑誌名

      Languages

      巻: 9 号: 1 ページ: 27-27

    • DOI

      10.3390/languages9010027

    • 関連する報告書
      2023 実施状況報告書
    • 査読あり / オープンアクセス / 国際共著
  • [雑誌論文] CAPTuring Accents: An Approach to Personalize Pronunciation Training for Learners with Different L1 Backgrounds2023

    • 著者名/発表者名
      Khaustova Veronica、Pyshkin Evgeny、Khaustov Victor、Blake John、Bogach Natalia
    • 雑誌名

      Lecture Notes in Computer Science

      巻: 14339 ページ: 59-70

    • DOI

      10.1007/978-3-031-48312-7_5

    • ISBN
      9783031483110, 9783031483127
    • 関連する報告書
      2023 実施状況報告書
    • 査読あり / 国際共著
  • [雑誌論文] Music and Choreography Metaphors in Spoken Language Rhythm Modelling and Their Application to Computer-Assisted Pronunciation Training for Mora-Timed Japanese2023

    • 著者名/発表者名
      Pyshkin Evgeny、Kusakari Akemi、Bogach Natalia、Blake John
    • 雑誌名

      The Kyoto Conference on Arts, Media & Culture 2023: Official Conference Proceedings

      巻: 2023 ページ: 497-511

    • DOI

      10.22492/issn.2436-0503.2023.43

    • 関連する報告書
      2023 実施状況報告書
  • [学会発表] Multimodal contextualizing and targeting exercises in iCAPT systems2024

    • 著者名/発表者名
      E. Pyshkin, J. Blake, V. Khaustova, V. Khaustov, I. Lezhenin, R. Svechnikov, and N. Bogach
    • 学会等名
      INTED-2024
    • 関連する報告書
      2023 実施状況報告書
    • 国際学会
  • [学会発表] Multimodal modeling of the mora-timed rhythm of Japanese and its application to computer-assisted pronunciation training2023

    • 著者名/発表者名
      E. Pyshkin, A. Kusakari, J. Blake, N. Ba Pham, and N. Bogach
    • 学会等名
      2023 14th IIAI International Congress on Advanced Applied Informatics
    • 関連する報告書
      2023 実施状況報告書
    • 国際学会
  • [学会発表] CAPTuring accents: An approach to personalize pronunciation Training for learners with different L1 backgrounds2023

    • 著者名/発表者名
      V. Khaustova, E. Pyshkin, V. Khaustov, J. Blake, and N. Bogach
    • 学会等名
      Speech and Computer 2023
    • 関連する報告書
      2023 実施状況報告書
    • 国際学会
  • [学会発表] Exploring Music and Choreographic Metaphors to Enhance Computer-Assisted Pronunciation Training in Mora-Timed Japanese2023

    • 著者名/発表者名
      E. Pyshkin, A. Kusakari, N. Bogach, and J. Blake
    • 学会等名
      The Kyoto Conference on Arts, Media & Culture 2023
    • 関連する報告書
      2023 実施状況報告書
    • 国際学会
  • [学会発表] Music. Languages. Computing2023

    • 著者名/発表者名
      E. Pyshkin
    • 学会等名
      University of Aizu Open Campus 2023
    • 関連する報告書
      2023 実施状況報告書
    • 招待講演
  • [学会・シンポジウム開催] Special Session on Information Management in Human-Centric Systems (IMHCS'23) as an event of 14th International Congress on Advanced Applied Informatics (IIAI AAI 2023)2023

    • 関連する報告書
      2023 実施状況報告書

URL: 

公開日: 2023-04-13   更新日: 2024-12-25  

サービス概要 検索マニュアル よくある質問 お知らせ 利用規程 科研費による研究の帰属

Powered by NII kakenhi