2020 Fiscal Year Research-status Report
Surrogate Model-based Integration Testing of CPS with Complex Black Box Components
| Project/Area Number |
20K23334
|
|---|
| Research Institution | National Institute of Informatics |
|---|
Principal Investigator |
KLIKOVITS Stefan 国立情報学研究所, アーキテクチャ科学研究系, 特任研究員 (10875983)
|
|---|
| Project Period (FY) |
2020-09-11 – 2022-03-31
|
| Keywords | Automated Driving / Search-Based Testing / Genetic Algorithms / Noisy Systems / Scenario Generation |
|---|
| Outline of Annual Research Achievements |
My goals for FY2020 were to gather a deeper understanding of Automated Driving Systems (ADS), how they could be used and how to verify their components.
I first focused on the Autonomoose simulator and established a close contact with its developers at the University of Waterloo, Canada. My plan to travel to Canada to learn there was cancelled due to Covid.
Our collaboration allowed me to develop a framework for ADS scenario creation and evaluation. I proved its capability by applying search-based testing (SBT), the theoretical results are in the process of publication.
Further, together with colleagues we participated in the SBST-Competition (https://sbst21.github.io/) where we developed an SBT tool for the BeamNG driving simulator. Our tool was among the top two successful ones, and best in result diversity.
I developed the knowledge and technical skills for successful research in ADS and plan on applying them. At the moment, I am pursuing two subprojects in the scope of Kakenhi. I am confident that both will result in publications.
|
| Current Status of Research Progress |
Current Status of Research Progress
3: Progress in research has been slightly delayed.
Reason
Due to Covid, my original plan to travel to the University of Waterloo, Canada was cancelled. Therefore, I could not have in-person meetings with the developers and directly learn from them about the Autonomoose Driving Simulator. To counteract, we established regular online meetings (roughly 1 hour every 2 weeks), which still allows me to learn from the developers. Thus, I now have a strong understanding and the technical skills to use the system. Nonetheless, the learning curve was slightly prolonged and as a result, I am minimally behind my original schedule.
On the positive side, I succeeded in adapting Autonomoose to cloud-based execution, such as Amazon Web Services. This allows me to run experiments in faster and in a highly parallel manner, which will allow me move more quickly in the future.
|
| Strategy for Future Research Activity |
Despite the pandemic and minor delays, my project is mostly on schedule.
My research advanced continuously and I managed to submit two initial publications that are in relation to my Kakenhi-project.
In the future, I want to continue following the plan and exploit the knowledge and technical scripts for ADS scenario evaluation. This will lead my research closer into the project of ADS component verification through scenario generation. Currently, I am currently working on identifying ADSs behaviour based on scenarios, as well as working on identifying the source components for noisy behaviour in ADSs. Both of these topics are of high interest and will hopefully lead to publications in good venues and journals.
Next to my Kakenhi-project work, I established collaborations with colleagues, whose feedback and discussions further advance my work.
|
| Causes of Carryover |
Due to the pandemic, my financial planning had to be slightly adapted. The part of the expenditures reserved for travelling (e.g. to conferences or to the University of Waterloo, Canada) remained unused.
The missing visit in Canada slowed my learning curve, also leading me to use less of my budget for cloud-based execution. Nonetheless, I managed to implement the cloud-execution framework and can now run experiments in a highly parallel manner (several dozens of machines in parallel).
Thus, I expect that in future, experiments will progress more quickly, and some expenses for cloud services and computing power will become active in FY2021 (instead of FY2020 as originally planned).
|
