• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Investigating the extent to which speech rater judgement relates to objective speech measures

Research Project

Project/Area Number 19K00931
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 02100:Foreign language education-related
Research InstitutionOsaka Jogakuin College (2021-2023)
Hiroshima Shudo University (2019-2020)

Principal Investigator

Head Philip (Philip Head)  大阪女学院大学, 国際・英語学部, 専任講師 (30801998)

Co-Investigator(Kenkyū-buntansha) 山根 典子  広島大学, 総合科学研究科, 准教授 (70319391)
CLENTON JONATHAN  広島大学, 総合科学研究科, 准教授 (80762434)
Project Period (FY) 2019-04-01 – 2025-03-31
Project Status Granted (Fiscal Year 2023)
Budget Amount *help
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2021: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2020: ¥2,080,000 (Direct Cost: ¥1,600,000、Indirect Cost: ¥480,000)
Fiscal Year 2019: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
KeywordsSpeech assessment / speech comprehensibility / speech assessment / Speech Comprehensibility / Speech Ratings / Listener backgrounds / Gender / Vocabulary / Phonology / Assessment / Speaking / Fluency
Outline of Research at the Start

2019 Prepare questionnaires, speech samples, and obtain rating data from Japan-based English L1 and L2 raters. Analyze speech sample phonological and lexical measures.
2020 Analysis of 2019 Japan-based data. Obtain data from English L1 and L2 raters unfamiliar with Japanese-accented English.

Outline of Annual Research Achievements

We completed data collection. In total 347 people signed up to participate in the study and 200 completed the procedures. Results of a post-evaluation questionnaire indicated that Japanese L1 students and teachers had significantly less confidence in their ability to rate English L2 speech than other rater groups. Also, a qualitative analysis of reasons for confidence found that Japanese L1 raters were mostly confident based on their effort and ability, whereas non-Japanese L1 raters were confident due to their training and experience as teachers as well as their exposure to different English varieties. Next, an analysis of different vocabulary measures and human ratings found that human ratings are heavily influenced by the number of words produced, so basic measures of lexical density correspond more closely to human ratings than measures that control for text length. Also, different rater groups show similar correlation patterns with lexical density measures, although severity of judgment may vary between groups. Finally, looking at factors affecting ratings, a RASCH analysis followed by Linear Mixed Effects modelling indicated that English L1 raters gave significantly higher ratings than non-English L1 raters. Also, Japan-based raters gave higher scores than Canada-based raters, but there were no differences between student and teacher raters. In addition, scores were lower when the rater and the speaker had the same L1, but higher when the rater lived in a country where the speaker L1 is primarily spoken.

Current Status of Research Progress
Current Status of Research Progress

3: Progress in research has been slightly delayed.

Reason

The project was slightly delayed due to difficulty recruiting sufficient numbers of Canada-based teachers to participate in the study. We now have finished collecting responses from this group and are proceeding with the final data analysis.

Strategy for Future Research Activity

We will complete the data analysis of our samples and submit the results to various journals for publication this year.

Report

(5 results)
  • 2023 Research-status Report
  • 2022 Research-status Report
  • 2021 Research-status Report
  • 2020 Research-status Report
  • 2019 Research-status Report
  • Research Products

    (9 results)

All 2023 2022 2021 2020 2019

All Journal Article (1 results) (of which Peer Reviewed: 1 results,  Open Access: 1 results) Presentation (8 results) (of which Int'l Joint Research: 5 results)

  • [Journal Article] The effect of shadowing and the ability of Japanese EFL learners to evaluate L2 English pronunciation2022

    • Author(s)
      Philip Head, Noriko Yamane
    • Journal Title

      大阪女学院大学紀要

      Volume: 18 Pages: 109-126

    • Related Report
      2021 Research-status Report
    • Peer Reviewed / Open Access
  • [Presentation] Effect of listener background on English L2 speech ratings.2023

    • Author(s)
      Head, P., & Yamane, N.
    • Organizer
      JALT2023 Growth Mindset in Language Education. Tsukuba, Japan.
    • Related Report
      2023 Research-status Report
    • Int'l Joint Research
  • [Presentation] Exploring the relationship between spoken lexical diversity scores and human rater vocabulary scores.2023

    • Author(s)
      Head, P., & Myint Maw, T. M.
    • Organizer
      The Association for Language Testing and Assessment of Australia and New Zealand (ALTAANZ) Conference 2023.
    • Related Report
      2023 Research-status Report
    • Int'l Joint Research
  • [Presentation] Confidence and challenges in Rating L2 speech: A comparative study of Japanese and Canadian Raters.2023

    • Author(s)
      Head, P., Yamane, N., & Clenton, J.
    • Organizer
      The 9th Annual International Conference of the Asian Association for Language Assessment. Tokyo, Japan
    • Related Report
      2023 Research-status Report
    • Int'l Joint Research
  • [Presentation] Student Reflections on Effectiveness of Shadowing and Listening Practice2021

    • Author(s)
      Philip Head, Noriko Yamane
    • Organizer
      Hiroshima JALT
    • Related Report
      2021 Research-status Report
  • [Presentation] Student Reflections on Effectiveness of Shadowing and Listening Practice2021

    • Author(s)
      Philip Head, Noriko Yamane
    • Organizer
      JALT
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Investigating the effects of listener background on L2 English speech ratings2020

    • Author(s)
      Philip Head, Noriko Yamane, Jon Clenton, & Masahiro Shinya.
    • Organizer
      Hiroshima Vocabulary Research Group
    • Related Report
      2020 Research-status Report
  • [Presentation] The effect of shadowing on Japanese EFL speaking ability and phonological awareness2020

    • Author(s)
      Philip Head & Noriko Yamane
    • Organizer
      Phonological Society of Japan Zoom conference
    • Related Report
      2020 Research-status Report
  • [Presentation] Comparing raters: Investigating the effects of listener background on English speech comprehensibility ratings.2019

    • Author(s)
      Philip Head, Noriko Yamane, Jon Clenton, & Masahiro Shinya.
    • Organizer
      7th British Council New Directions in English Language Assessment Conference, Yokohama, Japan.
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research

URL: 

Published: 2019-04-18   Modified: 2024-12-25  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi