Hierarchical Bayesian Item Response Theory Model for Reliable Peer Assessment

2016 Fiscal Year Final Research Report

• Project/Area Number: 15K16256
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Multi-year Fund
• Research Field: Educational technology
• Research Institution: The University of Electro-Communications
• Principal Investigator: Uto Masaki 電気通信大学, 大学院情報理工学研究科, 助教 (10732571)
• Project Period (FY): 2015-04-01 – 2017-03-31
• Keywords: ピアアセスメント / 項目反応理論 / eラーニング / eテスティング / 教育評価

Outline of Final Research Achievements

In peer assessment, a problem remains that reliability depends on the rater characteristics. For this reason, some item response models that incorporate rater parameters have been proposed. Those models are expected to improve the reliability if the model parameters can be estimated accurately. However, when applying them to actual peer assessment, the parameter estimation accuracy would be reduced for the following reasons. 1) The number of rater parameters increases rapidly because the models include higher-dimensional rater parameters. 2) The parameter estimation accuracy from sparse assessment data depends on hand-tuning parameters, called hyperparameters.
To solve the problems, we propose a new item response model for peer assessment that incorporates rater parameters to maintain as few rater parameters as possible. Furthermore, this article proposes a parameter estimation method using a hierarchical Bayes model for the proposed model that can learn the hyperparameters from data.

Free Research Field

教育工学