| 研究課題/領域番号 |
23K11062
|
|---|
| 研究種目 |
基盤研究(C)
|
| 配分区分 | 基金 |
|---|
| 応募区分 | 一般 |
|---|
| 審査区分 |
小区分60050:ソフトウェア関連
|
|---|
| 研究機関 | 国立情報学研究所 |
|---|
研究代表者 |
Arcaini Paolo 国立情報学研究所, アーキテクチャ科学研究系, 特任准教授 (50828118)
|
|---|
| 研究期間 (年度) |
2023-04-01 – 2026-03-31
|
| 研究課題ステータス |
交付 (2024年度)
|
| 配分額 *注記 |
4,550千円 (直接経費: 3,500千円、間接経費: 1,050千円)
2025年度: 1,170千円 (直接経費: 900千円、間接経費: 270千円)
2024年度: 1,300千円 (直接経費: 1,000千円、間接経費: 300千円)
2023年度: 2,080千円 (直接経費: 1,600千円、間接経費: 480千円)
|
|---|
| キーワード | differential testing / AI agents / reinforcement learning / artificial agents / explanation / Rank-Biased Overlap / comparison analysis / autonomous agents / testing / explainability / test generation |
|---|
| 研究開始時の研究の概要 |
Autonomous agents are used to test systems having complex requirements. Since these agents are based on reinforcement learning, the explanation of their test results is usually a challenge. The project will devise techniques to generate tests for autonomous agents, and to explain their results.
|
| 研究実績の概要 |
AI techniques have been successfully applied in various domains. Assessing the abilities of AI agents is challenging; indeed, it is difficult to establish rules to determine whether an AI agent's decision is correct or not. In this year, we proposed a differential testing approach that identifies tests in which an AI agent may perform a non-optimal move. The approach executes different AI agents over the same tests and checks whether they agree with each other. In case of disagreement, it assesses the level of disagreement. If two agents strongly disagree with each other, it is more likely that at least one agent made a wrong decision. The approach has been experimented on the Go game that, due to its complexity, has been taken as a benchmark to assess the abilities of AI agents.
|
| 現在までの達成度 |
現在までの達成度
2: おおむね順調に進展している
理由
The research proceeds as expected. The investigation conducted in second year shows that the research direction is promising, and good results have been obtained.
|
| 今後の研究の推進方策 |
The third year of the project will be focused on validating that the differential testing can indeed find useful scenarios. To do this, I will investigate to what extent the approach can be used to prioritize test cases. Such approach should be more efficient of other baseline approaches based, e.g., on random selection or other simple heuristics. Moreover, to assess the generalizability of the approach, I will investigate the application of the framework to other types of AI agents.
|
