Research and Development of Programs for Teacher Training to Improve Reliability in Evaluation

2004 Fiscal Year Final Research Report Summary

• Project/Area Number: 15530531
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Educaion
• Research Institution: National Institute for Educational Policy Research
• Principal Investigator: KAWAI Hisashi National Institute for Educational Policy Research, Dept.of Research Planning and Development, Deputy Director, 研究企画開発部, 企画調整官 (30214589)
• Co-Investigator(Kenkyū-buntansha): SAITO Michiko National Institute for Educational Policy Research, Educational Resources Research Center, Senior Researcher, 教育研究情報センター, 総括研究官 (40162234) ; HATOGAI Taro National Institute for Educational Policy Research, Dept.for Curriculum Research, Senior Researcher, 教育課程研究センター・基礎研究部, 総括研究官 (10280512)
• Project Period (FY): 2003 – 2004
• Keywords: Reliability in Evaluation / rubric / International Baccalaureate / Teacher Training / SAT

Research Abstract

Research was carried out focused on seeking measures to improve reliability in essay evaluation.
For the purpose of evaluating essays, the rubric has been developed in Europe and America as a yardstick for evaluation. According to reports, not only does making the same rubric available to both teachers and students improve the quality of the essay, it also ensures a certain degree of consistency in evaluation. However, this approach is not devoid of problems. This is because a rubric itself does not always ensure objective evaluation that is consistent throughout. It is therefore important for evaluators who utilize rubrics as part of evaluation to receive training.
In the United States, there is a test called the SAT, which has a major influence on university acceptance. From 2005 the SAT has been revised to include an essay. The researcher investigated the kinds of measures taken to improve reliability of evaluation for these essays. Additionally, the researcher looked at the diploma program of the International Baccalaureate Organization and chose the component TOK (Theory of Knowledge), which requires its students to write large volumes of essays, to investigate the kinds of tasks and themes chosen for these essays, the essay writing requirements and the method by which these essays were evaluated.
As an antecedent to training of evaluators in Japan, we believe that the above investigation serves as a reference to what kind of resources to prepare, and what kind of procedures to use in training so evaluators are equipped to perform objective evaluation that is consistent throughout.
To provide examples of what is successfully being incorporated into training, the NWREL essay "Communication Matters : A Workbook for Classroom Instruction and Assessment of the E.A.R. Oral Communication Traits" was translated and included as reference material.