2023 Fiscal Year Final Research Report
An Development of automated short-answer scoring system based on deep learning without using supervised scoring data
| Project/Area Number |
20H04300
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 62030:Learning support system-related
|
|---|
| Research Institution | The National Center for University Entrance Examinations |
|---|
Principal Investigator |
Ishioka Tsunenori 独立行政法人大学入試センター, 研究開発部, 教授 (80311166)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
中川 正樹 東京農工大学, 学内共同利用施設等, 特任教授 (10126295)
峯 恒憲 九州大学, システム情報科学研究院, 准教授 (30243851)
須鎗 弘樹 千葉大学, 大学院工学研究院, 教授 (70246685)
宮澤 芳光 独立行政法人大学入試センター, 研究開発部, 准教授 (70726166)
|
|---|
| Project Period (FY) |
2020-04-01 – 2023-03-31
|
| Keywords | 自然言語処理 / 自動採点 / 機械学習 / 深層学習 / トランスフォーマー / 手書き文字認識 |
|---|
| Outline of Final Research Achievements |
In recent years, research into deep learning methods called recurrent neural networks, especially transformers such as BART, has progressed, and their excellent performance has been proven. Here, we consider written sentences in natural language as time-series data with an order, and process this as input data. We attempted to process written response data from 120K common test trial surveys conducted in 2017 and 2017, from character recognition to automatic scoring using Bart, all at once. Our collaborative research group achieved an average agreement rate of 96% and a minimum of 93% in real-world operations without the manual training wheels used in conventional scoring systems. Additionally, by using a huge amount of data containing 60K questions for each question, we gained new knowledge about the sample size required for deep learning.
|
| Free Research Field |
情報数理
|
| Academic Significance and Societal Importance of the Research Achievements |
いままでの研究では学習データに用いるサンプルはせいぜい２千件程度であり、どの程度のサンプルがあれば十分な予測ができるかの目安は与えられていなかった。さらに九大グループでは意味的埋め込みと呼ばれる異なったアプローチによる方法を試みた。これら結果については本科研で3件の学会表彰（日本計算機統計学会第35回大会, 学生研究発表賞；Duolingo Award for IMPS 2021；SMASH22 Winter Symposium,準優秀賞）を受け、その成果については日本教育新聞や日経新聞教育面に大きく掲載された。その後、教育工学のトップ国際会議AIED 2022でも論文採択された。
|
