| 研究実績の概要 |
My research aims to develop an argument-oriented writing assistant for EFL students. The tool extracts and visualises the argumentative structure in essays. Based on the predicted argumentative structure, the tool also presents suggested improvements of the essays by reordering sentences.
Currently, I have created a corpus of 434 annotated EFL essays with argumentative structure and sentence reordering. This is the largest annotated EFL essays so far. I also developed novel inter-annotator agreement metrics for a better evaluation of annotation quality. The paper describing this corpus has been accepted at an international journal with minor revision and will be available in the next fiscal year. Alongside this dataset annotation project, I also developed an in-house annotation tool TIARA to accommodate our task (publicly available). The paper describing this tool has been published at an international conference.
Using the annotated corpus, I have built a deep learning model for parsing argumentative structure in essays. Given an essay (a sequence of sentences) as input, this module predicts the argumentative relations between sentences, forming a hierarchical structure. I employed a “Biaffine-Attention-Model”, which is the state-of-the-art model for automatic parsing of hierarchical structure. The paper describing automatic parsing experiment has been presented at an international conference as well.
|
| 現在までの達成度 (区分) |
現在までの達成度 (区分)
2: おおむね順調に進展している
理由
In this fiscal year, I annotated argumentative essays sampled from the ICNALE corpus (writings by Asian students). There are two annotation layers: (1) argumentative structure and (2) sentence reordering. I confirmed that the argumentative structure analysis could be very useful for textual assessment. The sentence reordering procedure also helped in improving essays.
To realise my goal in developing a writing assistant tool, I then experimented on automatic argumentative structure parsing models using deep learning techniques. The model has not achieved an industrial-level performance yet, but this experiment provided insights and contributions for work on essays written by non-native English speakers (which is a niche research area).
|
| 今後の研究の推進方策 |
The automatic parsing model is very important for the downstream sentence reordering task. To this end, I will still try various techniques to improve the parsing performance in the next fiscal year. Particularly, using multi-task learning technique and training-data expansion. Multi-task learning will help the model to perform better. As my corpus is relatively small for a deep learning experiment, expanding the data using other corpora will be helpful.
I will also implement the second module, i.e., sentence reordering. Given an essay and its argumentative structure as input, this module rearranges sentences in the essay to improve text coherence. There are two subtasks involved in this module: (a) pairwise ordering constraint prediction and (b) inference. In the pairwise ordering constraint prediction task, I determine the partial ordering preference between sentences connected by argumentative relations. This subtask outputs a topological graph. In the inference task (b), I then reorder sentences in the text based on the topological graph of (a). Since there is no quantitative metrics to measure whether the reordered essay is better than the original ones written by students, I plan to evaluate the system outputs by manual evaluation in the next fiscal year.
|
