2023 Fiscal Year Research-status Report
Forward-Looking Argument Mining
| Project/Area Number |
23K16956
|
|---|
| Research Institution | National Institute of Advanced Industrial Science and Technology |
|---|
Principal Investigator |
Chen ChungーChi 国立研究開発法人産業技術総合研究所, 情報・人間工学領域, 研究員 (60971405)
|
|---|
| Project Period (FY) |
2023-04-01 – 2025-03-31
|
| Keywords | forward-looking argument / scenario planning / claim generation / finance / argument mining |
|---|
| Outline of Annual Research Achievements |
We construct a forward-looking argument mining dataset based on analysis reports and also explore the potential of pre-trained language models and large language models. The first shared task is finished with the proposed dataset, and we are now organizing the second iteration, which focuses on the impact period. The first shared task, FinArg-1 in NTCIR, attracted 11 research teams worldwide to join us. The baselines for scenario generation and forward-looking claim generation are also explored.
|
| Current Status of Research Progress |
Current Status of Research Progress
1: Research has progressed more than it was originally planned.
Reason
The progress of this project is ahead of schedule. First, we already finished the first explorations on scenario generation and forward-looking claim generation (expected). We also obtained some human evaluation results and compared them with LLM evaluation results (ahead). The first shared task based on the proposed dataset is also finished (ahead). Moreover, we are now organizing a workshop, named the 1st Agent AI for Scenario Planning, at IJCAI-2024. It is an achievement that is not promised in the proposal, but we think that it is an important way to gather researchers to discuss the proposed topic of this project.
|
| Strategy for Future Research Activity |
Overall, we are going to wrap up our experimental results and add more experiments for the proposed datasets. We'll focus on the model construction and evaluation in FY2024. First, the impact duration of analysis reports and news articles will be discussed. Second, some approaches for improving forward-looking claim generation will be proposed. Third, the second iteration of the shared task will be organized in NTCIR-18, and the workshop related to scenario planning will be organized in IJCAI-24. Discussions on evaluating forward-looking arguments will be held, including with LLMs and human evaluators. Finally, we will wrap up our project and share it in ECAI-24 as a tutorial.
|
| Causes of Carryover |
Because we still have a very small amount that was not used this year for LLM's API fee, we plan to use it in the following year. This amount will be used to achieve our goal of designing evaluation methods for the generated text.
|
