2022 Fiscal Year Research-status Report
Validating a CEFR-informed Open Access English Placement Test
| Project/Area Number |
21K13083
|
|---|
| Research Institution | Tezukayama University |
|---|
Principal Investigator |
Bower Jack 帝塚山大学, 全学教育開発センター, 准教授 (90406687)
|
|---|
| Project Period (FY) |
2021-04-01 – 2025-03-31
|
| Keywords | language testing / assessment / CEFR / CEFR-J / course placement / Moodle |
|---|
| Outline of Annual Research Achievements |
In 2022 a slightly revised version of the course placement test with a 40-minute time limit was trailed. 247 of the Japanese university students who took the test on their smartphones through the Moodle LMS, gave permission for their test results to be used for research purposes. As with the first pilot, strong statistical evidence from a Cronbach's alpha of 0.9 and a Rasch person separation statistic of 2.91 was found to support the test function of dividing students into three ability groups for course placement purposes. However, item analysis revealed some problems with item quality and with the ordering of test items when ranked by CEFR-J wordlist levels. Therefore, further revision and creation of test items is needed before collecting criterion validity data and setting cut scores.
|
| Current Status of Research Progress |
Current Status of Research Progress
3: Progress in research has been slightly delayed.
Reason
Two major challenges were encountered in the second year of this research project. Firstly, although the test was used for actual course placement purposes at the researcher's university for the first time at that start of the 2023 academic year, only reading items could be used and listening items had to be abandoned due to administrative constraints. Furthermore, permission was not granted by the university administration for the use of student test data from the the first actual implementation of the test for research purposes. Secondly, it was found that some test items need further revision, so the goal of correlating the test with results from commercial English placement tests is better left until the 2024 academic year after test items have been further revised.
|
| Strategy for Future Research Activity |
In the 2023 academic year the researcher will focus on building a larger question bank for the placement test, from which to select well-functioning items for the 2024 iteration of the test. The new test items will be trialed with Japanese university students at the researcher:s institution and the results will be equated with results from the 2022 pilot test administration.The researcher will also attempt to get permission from his university to be able to use results from the 2024 administration the test for English course placement at his university for research purposes. Additionally, the researcher will submit research papers on this project to at least two academic journals, and will again endeavor to present research results at one domestic and one international conference.
|
| Causes of Carryover |
In the next fiscal year I plan to present at one domestic and one international conference. English native speaker voice actors will also need to be hired for recording new and revised listening test questions. In addition, research funds will be used download research articles and to purchase books for literature review for writing research articles.
|
