| Project/Area Number |
23K00679
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 02100:Foreign language education-related
|
|---|
| Research Institution | The University of Aizu |
|---|
Principal Investigator |
Pyshkin Evgeny 会津大学, コンピュータ理工学部, 上級准教授 (50794088)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
Mozgovoy Maxim 会津大学, コンピュータ理工学部, 上級准教授 (60571776)
BLAKE John 会津大学, コンピュータ理工学部, 上級准教授 (80635954)
|
|---|
| Project Period (FY) |
2023-04-01 – 2026-03-31
|
| Project Status |
Granted (Fiscal Year 2023)
|
| Budget Amount *help |
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2025: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2024: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2023: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
|
|---|
| Keywords | CAPT feedback / personalization / targeting / multimodality / multilingual CAPT / intelligent CAPT / multimodal / intonation / pitch evaluation / speech visualization |
|---|
| Outline of Research at the Start |
This project consists of three core work packages: Formulation of theoretical grounds for using dynamic system models in CAPT systems; Enhancing the CAPT environment with attitudinal and contrastive pronunciation exercises; Refining mobile components UI with respect to evaluation of speech dynamics.
|
| Outline of Annual Research Achievements |
The focus of works in AY2023 was on developing models, interfaces, and software to improve CAPT feedback with respect to supporting contrastive pronunciation in context and investigating the possibilities of accent recognition for personalizing the CAPT exercises and feedback addressing the learners' L1 background.
Studies on multimodal modeling of mora-timed Japanese enhanced our CAPT environment towards its multilingual setup. We published an extensive study of the important aspects of multilingual multimodality and the specific language features affecting the architecture and tools of an intelligent CAPT environment. By selecting one language from each of three different language families, we demonstrated how a single environment may be tailored to cater for
different target languages. We demonstrated that since the applied underlying mathematical and phonological models, as well as the feedback production algorithms, are based on sound signal processing and modeling rather than on particular languages, the system is language-agnostic and serves as an open toolkit for developing phrasal intonation training exercises for an open selection of languages. We succeeded to describe our response to the challenges in visualizing and segmenting recorded pitch signals and modeling the language melody and rhythm necessary for such a multilingual adaptation, particularly for tonal syllable-timed and mora-timed languages.
We achieved the interesting cross-disciplinary results on connecting the CAPT feedback models to other learners' backgrounds including music, languages, and arts.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The experiments on accent classification using Speech Accent Archive and CNN enabled the possibilities to define a model for CAPT exercise targeting depending on learners' L1. Empirical studies and assessment of suggest feedback tailoring models and interfaces created good grounds for contribution to the theory of dynamic assessment and mediation for moving learners beyond the plateau (to be published in AY2024).
|
| Strategy for Future Research Activity |
We need further efforts to integrate the features assessed in scope of research versions of learning software into the intelligent CAPT environment. Assessment of existing tool in the classroom and by individual learners will help to better understand what are the most productive models for user feedback personalization. Also, we will describe our findings on how to contextualize the speech situations using a variety of CAPT feedback production forms and what is the role of iCAPT systems in enabling learners to move beyond their learning plateau. We will rely on the concepts of the sociocultural theory, such as mediation through the zone of proximal development, thus connecting the pedagogical, linguistic, and computational aspects of CAPT design and development.
|
