Development and Evaluation of Ordered Multiple-Choice Items with Distractors at Various Difficulty Levels

2023 Fiscal Year Final Research Report

• Project/Area Number: 20K14177
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 10020:Educational psychology-related
• Research Institution: The National Center for University Entrance Examinations
• Principal Investigator: Terao Takahiro 独立行政法人大学入試センター, 研究開発部, 准教授 (70827055)
• Project Period (FY): 2020-04-01 – 2024-03-31
• Keywords: 順序多肢選択式項目 / 誤答選択肢 / 系統的錯乱枝生成法 / 選択肢リスト

Outline of Final Research Achievements

This study developed a new data visualization methodology called distractor cascade analysis, which makes it possible to show selection patterns across ordered multiple-choice items. This work won the 2022 EM:IP cover graphic award and was acknowledged to the international practitioners.
Study 1 aimed to experiment ordered multiple-choice items with various difficulty level. A total of 227 highschool students answered the test and results showed that the intended difficulty ordering was observed only in three of 12 items. It was revealed that a set of experimental items in the study was much difficult for Japanese highschool students.

Free Research Field

教育測定学

Academic Significance and Societal Importance of the Research Achievements

本研究課題では，古典的な項目分析ではとらえきれなかった，複数問題にわたる選択パターンを可視化する手法を開発し，実データを適用した事例を創出した。これまで1問1問に対する統計的分析にとどまっていたところ，複数個の問題にわたる選択状況を把握する可視化手法を開発したことにより，同一難易度水準の選択肢が連続して選ばれているか等の確認を行える点が画期的であり，学術的意義が高い。この特長は，順序多肢選択式問題の品質を評価する上でとりわけ有用な統計図表となり，統計に必ずしも明るくない者にも解釈可能な図となっている点は，実用性も高い。