2017 Fiscal Year Research-status Report
Developing a learner-adaptive captioning system to improve second language listening
| Project/Area Number |
17K02925
|
|---|
| Research Institution | Kyoto University |
|---|
Principal Investigator |
MESHGI Kourosh 京都大学, 情報学研究科, 特定研究員 (80774835)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
Mirzaei Maryam 国立研究開発法人理化学研究所, 革新知能統合研究センター, 特別研究員 (10810509)
|
|---|
| Project Period (FY) |
2017-04-01 – 2020-03-31
|
| Keywords | Data-driven / automatic scaffolding / NLP |
|---|
| Outline of Annual Research Achievements |
The baseline of the system has been designed, implemented and pilot tested with the state-of-the-art technologies having stability and accuracy in mind. The system is ready to produce different variations, test new features, integrate machine learning techniques and improve the quality of the caption by tweaking the feature extractors.
We also submitted two manuscripts to top-rank international CALL conferences, both accepted for oral presentation.
K. Meshgi, M. S. Mirzaei, “A Data-driven Approach to Generate Partial and Synchronized Caption for Second Language Listeners,” in Proc. of WorldCALL’18, Concepcion, Chile, Nov 2018.
M. S. Mirzaei, K. Meshgi, “Automatic Scaffolding for L2 Listeners by Leveraging Natural Language Processing,” in Proc. of EuroCALL’18, Jyvaskyla, Finland, Aug 2018.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
According to the timing of the project, by now the baseline should be ready and a pilot study should be conducted. We are strictly following the predicted timeline, and have no difficulty pursuing it. The budget would be used to register to EuroCALL and WorldCALL conferences for the accepted papers, and to present the achievements of this project to the CALL community.
|
| Strategy for Future Research Activity |
As planned, the next steps of the study entail more learner personalization, system content generation, online system release, and different experiments including eye-tracking tests, longitudinal test, and if possible to collaborate with relevant researchers, to incorporate brain studies on the effect of the proposed caption on the brain of the learners. This round of research requires new equipment, and to outsource the software to a 3rd party software firm to develop the system in a user-friendly and stable manner.
|
| Causes of Carryover |
The estimate for the previous financial year was to buy all equipment at the beginning of the project. However, we decided to buy the equipment at the beginning of each phase of the experiment to procure the lasts technology. Therefore we delayed the purchase of the equipment to this financial year before the beginning of the next phase of experiments.
In this phase of research requires new equipment, and to outsource the software to a 3rd party software firm to develop the system in a user-friendly and stable manner.
|
