2022 Fiscal Year Research-status Report
Evaluating communicative adequacy in English speaking - development and validation of rater evaluation rubric
| Project/Area Number |
21K00786
|
|---|
| Research Institution | Osaka University |
|---|
Principal Investigator |
リー シーチェン・ナンシー 大阪大学, 大学院人文学研究科(言語文化学専攻), 准教授 (60512308)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
小川 知恵 京都産業大学, 文化学部, 准教授 (40645107)
|
|---|
| Project Period (FY) |
2021-04-01 – 2025-03-31
|
| Keywords | communicative adequacy / speaking development / speaking proficiency / speaking evaluation |
|---|
| Outline of Annual Research Achievements |
The present research aims to develop and validate new evaluation rubrics for measuring English speaking communicative adequacy. It conceptualized communicative adequacy by asking participants to evaluate monologues and conversations. From the evaluation of monologues, six criteria were developed: intonation or pronunciation; content and meaning conveyed; grammar; vocabulary, fluency, and listener perspective. From dialogues, seven evaluation criteria were created: fluency, attitude, pronunciation & sound, organization, vocabulary & grammar, understanding interlocutor, content and interactive task assessment. Results of the up-to-date research were presented at both domestic and international conferences. A paper was published by the RELC Conference in Anthology Series.
|
| Current Status of Research Progress |
Current Status of Research Progress
3: Progress in research has been slightly delayed.
Reason
Student participants were asked to rate utterances intuitively and to give reasons to justify their rating. In the first year, a total of 53 Japanese university students initiatively evaluate recorded English monologues (using five-point Likert scale). After participants rated the narrations, they were asked to justify their ratings using an open-ended survey. In the second year, 58 university students evaluated conversations and were also asked to justify their evaluations. Almost 4000 words were collected in second year. The comments were grouped into categories. Seven evaluation criteria were created after coding all comments: fluency, attitude, pronunciation & sound, organization, vocabulary & grammar, understanding interlocutor, content and interactive task assessment.
|
| Strategy for Future Research Activity |
The present research will investigate teacher raters’ conceptualization of communicative adequacy. It aims to engage teacher raters for doing intuitive evaluation of pre-recorded cartoon narrations (collected by the principal researcher for her PhD research). Validation of rater rating will be conducted. In this stage of research, Rasch analysis will be conducted to validate rater ratings of the CAF measurement group. Work-in-progress research will be presented at domestic and international conferences.
|
| Causes of Carryover |
Conference presentations became online.
|
