Development of Automated Essay-Scoring System for a Task-Based Writing Test
| Project/Area Number |
16K02981
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Research Field |
Foreign language education
|
|---|
| Research Institution | Meiji Gakuin University |
|---|
Principal Investigator |
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
石井 雄隆 早稲田大学, 大学総合研究センター, 助手 (90756545)
|
|---|
| Project Period (FY) |
2016-04-01 – 2020-03-31
|
| Project Status |
Completed (Fiscal Year 2019)
|
| Budget Amount *help |
¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000)
Fiscal Year 2018: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2017: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2016: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
|
|---|
| Keywords | ライティング / 自動評価採点 / ライティングテスト / 自動採点 / 信頼性 / 妥当性 / ライティング・テスト / 教育評価・測定 / 自動 |
|---|
| Outline of Final Research Achievements |
This study focuses on the reliability and validity of an automated essay-scoring system for a task-based writing test. The system was revised and 150 second-year high school students participated in the trial of the system and the Accuracy and Communicability values were calculated by the resulting formulas. To estimate the degree to which the indices were collectively related to the prediction of the scores, correlation analyses were conducted. The results showed that moderately high correlation existed between the scores of the both tasks and their indices. To validate the predictions of the formulas, the values were compared to the Criterion scores of the high school students. Correlations between the Accuracy and Communicability values were significant. The composite values of the two tasks were also significantly correlated. The results of the questionnaire evaluating scores of the tasks showed that the scores were acceptable and appropriate for the students.
|
| Academic Significance and Societal Importance of the Research Achievements |
Accuracy、 Communicabilityの予測得点とCriterion スコアとの相関係数により、基準関連妥当性の検証を行ったところ比較的強い相関が見られ、また両者の合計点をTBWT によって測定されるライティング知識体系の総和 (総合評価) と位置づけて相関分析を行ったところ、強い相関が見られた。これらの結果から、自動採点結果はCriterion のパフォーマンスを一定程度反映していることが確認された。さらに、評価結果に対するアンケート調査により結果妥当性の検証を試みたところ、評価結果が利害関係者である高校生に与える影響は適切であったことが確認された。
|
Report
(5 results)
Research Products
(10 results)
