Validating a CEFR-informed Open Access English Placement Test
| Project/Area Number |
21K13083
|
|---|
| Research Category |
Grant-in-Aid for Early-Career Scientists
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
Basic Section 02100:Foreign language education-related
|
|---|
| Research Institution | Tezukayama University |
|---|
Principal Investigator |
Bower Jack 帝塚山大学, 全学教育開発センター, 准教授 (90406687)
|
|---|
| Project Period (FY) |
2021-04-01 – 2025-03-31
|
| Project Status |
Granted (Fiscal Year 2022)
|
| Budget Amount *help |
¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000)
Fiscal Year 2024: ¥260,000 (Direct Cost: ¥200,000、Indirect Cost: ¥60,000)
Fiscal Year 2023: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Fiscal Year 2022: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Fiscal Year 2021: ¥130,000 (Direct Cost: ¥100,000、Indirect Cost: ¥30,000)
|
|---|
| Keywords | language testing / assessment / CEFR / CEFR-J / course placement / Moodle / Language testing / Placement test / Test validation / English proficiency test |
|---|
| Outline of Research at the Start |
This research is a validation study of an online, CEFR-informed, English course placement test designed for Japanese university students. Results of this research will be of benefit to language educators seeking to make their own online, CEFR-informed or CEFR-linked language tests.
|
| Outline of Annual Research Achievements |
In 2022 a slightly revised version of the course placement test with a 40-minute time limit was trailed. 247 of the Japanese university students who took the test on their smartphones through the Moodle LMS, gave permission for their test results to be used for research purposes. As with the first pilot, strong statistical evidence from a Cronbach's alpha of 0.9 and a Rasch person separation statistic of 2.91 was found to support the test function of dividing students into three ability groups for course placement purposes. However, item analysis revealed some problems with item quality and with the ordering of test items when ranked by CEFR-J wordlist levels. Therefore, further revision and creation of test items is needed before collecting criterion validity data and setting cut scores.
|
| Current Status of Research Progress |
Current Status of Research Progress
3: Progress in research has been slightly delayed.
Reason
Two major challenges were encountered in the second year of this research project. Firstly, although the test was used for actual course placement purposes at the researcher's university for the first time at that start of the 2023 academic year, only reading items could be used and listening items had to be abandoned due to administrative constraints. Furthermore, permission was not granted by the university administration for the use of student test data from the the first actual implementation of the test for research purposes. Secondly, it was found that some test items need further revision, so the goal of correlating the test with results from commercial English placement tests is better left until the 2024 academic year after test items have been further revised.
|
| Strategy for Future Research Activity |
In the 2023 academic year the researcher will focus on building a larger question bank for the placement test, from which to select well-functioning items for the 2024 iteration of the test. The new test items will be trialed with Japanese university students at the researcher:s institution and the results will be equated with results from the 2022 pilot test administration.The researcher will also attempt to get permission from his university to be able to use results from the 2024 administration the test for English course placement at his university for research purposes. Additionally, the researcher will submit research papers on this project to at least two academic journals, and will again endeavor to present research results at one domestic and one international conference.
|
Report
(2 results)
Research Products
(8 results)
