2022 Fiscal Year Research-status Report
Development of an open-source speaking evaluation platform with automated scoring
| Project/Area Number |
20K00807
|
|---|
| Research Institution | Kochi University of Technology |
|---|
Principal Investigator |
ダニエルズ ポール 高知工科大学, 共通教育教室, 教授 (50307245)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
熊井 信弘 学習院大学, 付置研究所, 教授 (00248999)
|
|---|
| Project Period (FY) |
2020-04-01 – 2024-03-31
|
| Keywords | speaking / writing / language / assessment / AI |
|---|
| Outline of Annual Research Achievements |
With rapid advancements in AI, the past year has proved to be a pivotal one for language recognition and assessment. Using a variety of newly available AI engines, we were able to provide language learners with more specific and more personalized grammar and content feedback. First, using 'Whisper', OpenAI's speech-to-text engine, we were able to successfully generate accurate text transcripts from students speaking activities, including group presentations. The transcribed text was then fed to both 'DeepL Write' and 'ChatGPT' using APIs. These AI engines were able to check for grammar and punctuation mistakes in the language that was produced by the students, and even rephrase sentences.
Using the feedback generated by the AI engines, we were also able to establish a simple rubric to automatically score spoken language tasks. Secondly, we were able to integrate OpenAI's ChatGPT with our current speaking platform, allowing language learners to practice their spoken conversation skills with a Chatbot.
Finally, we were able to successfully implement auto-scored speaking tasks into our first-year placement test. The data collected from the speaking tasks were used to analyze the correlations between speaking scores and listening and reading scores from a standardized placement test. The results of the analysis were published in academic journals.
|
| Current Status of Research Progress |
Current Status of Research Progress
1: Research has progressed more than it was originally planned.
Reason
Now that travel restrictions have eased, it is easier to attend academic meetings and this past year we were able to meet with language teachers in person rather than online to share ideas and our research findings. This recent face-to-face interaction proved to be a motivating factor for our research. Unfortunately, other global issues have slightly impacted our research. Since our core project programmer resides in Russia, it was challenging work together due to the global sanction on Russian banks. But perhaps the most promising impetus for our research this past year was the rapid advancements made processing language using AI.
|
| Strategy for Future Research Activity |
This year we plan to test our latest tool for speaking practice, which allows language learners to practice their spoken conversation skills with an AI Chatbot. Using the language data gathered from the spoken conversations, our spoken assessment engine will be updated to automatically generate scores for the speaking tasks.
|
| Causes of Carryover |
Due to the pandemic travel restrictions, it proved challenging to travel to academic meetings. In addition, the Ukrainian conflict prevented us from making payments to our lead programmer who is based in Russia.
In 2024, we are eager to present our research at academic conferences now that travel restrictions have eased. We will also focus on providing learners with more authentic online speaking practice via the use of recent AI engines such as OpenAI's ChatGPI and Google's BARD.
|
