• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of Automated Essay-Scoring System for a Task-Based Writing Test

Research Project

Project/Area Number 16K02981
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Foreign language education
Research InstitutionMeiji Gakuin University

Principal Investigator

Sugita Yoshihito  明治学院大学, 文学部, 教授 (70363885)

Co-Investigator(Kenkyū-buntansha) 石井 雄隆  早稲田大学, 大学総合研究センター, 助手 (90756545)
Project Period (FY) 2016-04-01 – 2020-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000)
Fiscal Year 2018: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2017: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2016: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Keywordsライティング / 自動評価採点 / ライティングテスト / 自動採点 / 信頼性 / 妥当性 / ライティング・テスト / 教育評価・測定 / 自動
Outline of Final Research Achievements

This study focuses on the reliability and validity of an automated essay-scoring system for a task-based writing test. The system was revised and 150 second-year high school students participated in the trial of the system and the Accuracy and Communicability values were calculated by the resulting formulas. To estimate the degree to which the indices were collectively related to the prediction of the scores, correlation analyses were conducted. The results showed that moderately high correlation existed between the scores of the both tasks and their indices. To validate the predictions of the formulas, the values were compared to the Criterion scores of the high school students. Correlations between the Accuracy and Communicability values were significant. The composite values of the two tasks were also significantly correlated. The results of the questionnaire evaluating scores of the tasks showed that the scores were acceptable and appropriate for the students.

Academic Significance and Societal Importance of the Research Achievements

Accuracy、 Communicabilityの予測得点とCriterion スコアとの相関係数により、基準関連妥当性の検証を行ったところ比較的強い相関が見られ、また両者の合計点をTBWT によって測定されるライティング知識体系の総和 (総合評価) と位置づけて相関分析を行ったところ、強い相関が見られた。これらの結果から、自動採点結果はCriterion のパフォーマンスを一定程度反映していることが確認された。さらに、評価結果に対するアンケート調査により結果妥当性の検証を試みたところ、評価結果が利害関係者である高校生に与える影響は適切であったことが確認された。

Report

(5 results)
  • 2019 Annual Research Report   Final Research Report ( PDF )
  • 2018 Research-status Report
  • 2017 Research-status Report
  • 2016 Research-status Report
  • Research Products

    (10 results)

All 2019 2018 2017 2016 Other

All Journal Article (3 results) (of which Peer Reviewed: 3 results,  Acknowledgement Compliant: 1 results) Presentation (3 results) (of which Int'l Joint Research: 1 results) Remarks (4 results)

  • [Journal Article] Reliability and Validity of an Automated Essay-Scoring System for a Task-Based Writing Test2019

    • Author(s)
      Yoshihito SUGITA
    • Journal Title

      全国英語教育学会紀要 (ARELE)

      Volume: 30 Pages: 17-32

    • NAID

      130007826118

    • Related Report
      2018 Research-status Report
    • Peer Reviewed
  • [Journal Article] タスクに基づくライティングテスト自動採点システムにおける客観的評価指標の検討(2)2017

    • Author(s)
      杉田由仁 石井雄隆
    • Journal Title

      第43回全国英語教育学会島根研究大会発表予稿集

      Volume: 1 Pages: 178-179

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Validation of an Automated Essay-Scoring System for Task-Based Writing Tests2017

    • Author(s)
      Yoshihito Sugita and Yutaka Ishii
    • Journal Title

      Information Communication Technology Practice and Research 2016

      Volume: 印刷中

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Presentation] タスクに基づくライティング・テスト自動評価採点システム信頼性・妥当性の検証2018

    • Author(s)
      杉田由仁
    • Organizer
      第44回全国英語教育学会京都研究大会
    • Related Report
      2018 Research-status Report
  • [Presentation] タスクに基づくライティングテスト自動採点システムにおける客観的評価指標の検討(2)2017

    • Author(s)
      杉田由仁 石井雄隆
    • Organizer
      第43回全国英語教育学会島根研究大会
    • Related Report
      2017 Research-status Report
  • [Presentation] Objective rating indices to automated essay-scoring systems for writing assessment2016

    • Author(s)
      Yoshihito Sugita
    • Organizer
      The 50th IATEFL Annual Conference
    • Place of Presentation
      The International Convention Centre (Birmingham, UK)
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Remarks] タスクに基づくライティング・テスト自動採点サイト

    • Related Report
      2019 Annual Research Report
  • [Remarks] 明治学院大学英語教授法(TESOL)研究室

    • Related Report
      2018 Research-status Report
  • [Remarks] タスクに基づくライティング・テスト自動採点サイト

    • URL

      http://www.meijigakuin.ac.jp/~ysugita/

    • Related Report
      2017 Research-status Report
  • [Remarks] 明治学院大学文学部 英語教授法研究室

    • URL

      http://www.meijigakuin.ac.jp/~ysugita/homepage/

    • Related Report
      2016 Research-status Report

URL: 

Published: 2016-04-21   Modified: 2021-02-19  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi