Investigating the extent to which speech rater judgement relates to objective speech measures
Project/Area Number |
19K00931
|
Research Category |
Grant-in-Aid for Scientific Research (C)
|
Allocation Type | Multi-year Fund |
Section | 一般 |
Review Section |
Basic Section 02100:Foreign language education-related
|
Research Institution | Osaka Jogakuin College (2021-2022) Hiroshima Shudo University (2019-2020) |
Principal Investigator |
Head Philip (Philip Head) 大阪女学院大学, 国際・英語学部, 専任講師 (30801998)
|
Co-Investigator(Kenkyū-buntansha) |
山根 典子 広島大学, 総合科学研究科, 准教授 (70319391)
CLENTON JONATHAN 広島大学, 総合科学研究科, 准教授 (80762434)
|
Project Period (FY) |
2019-04-01 – 2024-03-31
|
Project Status |
Granted (Fiscal Year 2022)
|
Budget Amount *help |
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2021: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2020: ¥2,080,000 (Direct Cost: ¥1,600,000、Indirect Cost: ¥480,000)
Fiscal Year 2019: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
|
Keywords | speech comprehensibility / speech assessment / Speech Comprehensibility / Speech Ratings / Listener backgrounds / Gender / Vocabulary / Phonology / Assessment / Speaking / Fluency |
Outline of Research at the Start |
2019 Prepare questionnaires, speech samples, and obtain rating data from Japan-based English L1 and L2 raters. Analyze speech sample phonological and lexical measures. 2020 Analysis of 2019 Japan-based data. Obtain data from English L1 and L2 raters unfamiliar with Japanese-accented English.
|
Outline of Annual Research Achievements |
In 2022 we concluded collecting data from Japan-based raters and submitted applications for permission to collect responses from Canada-based teacher raters at public schools. We also began quantitative analysis of data collected so far. Initial findings suggest that the L1 Japanese teachers of English rated the L1 Japanese L2 English speaker samples as low scoring for all categories aside from overall comprehensibility, but teaching experience did not influence L1 Japanese L2 English speaker sample ratings. However, for the L1 Thai L2 English samples, rater L1 only influenced segmental pronunciation and we observed a positive influence of teaching experience, with the L1 Japanese L2 English students rating samples significantly higher than their English teacher counterparts. In addition, both Japanese teachers and students report significantly lower confidence in their ability to rate English speech than English L1 or L2 English teachers in Japan. Furthermore, regarding evaluating specific aspects of speech, fluency was the easiest, while grammar and lexis were often the most challenging. For English L2 speakers, there was a lack of confidence in their own knowledge of correct grammar and vocabulary, while for English L1 speakers, the difficulty was in judging the complexity of the grammar and lexis accurately based on the sample length. In terms of vocabulary analysis, we observed that there was correlation between self-assessment vocabulary scores and some lexical measures, but no correlations were found for ratings by outside raters.
|
Current Status of Research Progress |
Current Status of Research Progress
3: Progress in research has been slightly delayed.
Reason
In 2022 we concluded collecting data from Japan-based raters and submitted applications for permission to collect responses from Canada-based teacher raters. We also began quantitative analysis of vocabulary measures of the speech samples for comparison with human vocabulary ratings. The difficulties in getting official permission to contact teachers in Canada delayed the final gathering of data. However, we have applied to multiple Canadian boards of education for research approval and hope to officially contact teachers and conclude data collection by September 2023. We have also applied to several international conferences to present our findings.
|
Strategy for Future Research Activity |
This year we plan to complete all data collection and analysis. We will finish comparing ratings of Japan-based raters with their Canadian counterparts. We also plan to finish obtaining objective measures of speech sample fluency, vocabulary, and phonological features to compare to the human ratings. We have also applied to present our findings at the 2023 JALT International conference. We are also in the process of writing a paper on our initial findings that we will submit this summer and then we will write further papers once the final data collection and analysis is completed.
|
Report
(4 results)
Research Products
(6 results)