2023 Fiscal Year Research-status Report
Emergent Reality: Knowledge Formation from Multimodal Learning through Human-Robot Interaction in Extended Reality
| Project/Area Number |
22K17981
|
|---|
| Research Institution | Ritsumeikan University |
|---|
Principal Investigator |
ElHafi Lotfi 立命館大学, 総合科学技術研究機構, 准教授 (90821554)
|
|---|
| Project Period (FY) |
2022-04-01 – 2025-03-31
|
| Keywords | Extended Reality / Human-Robot Interaction / Multimodal Learning |
|---|
| Outline of Annual Research Achievements |
Significant progress has been made in human-robot interactive learning within extended reality with two main achievements: 1) a mixed reality-based 6D-pose annotation system for robot manipulation in service environments, enhancing the accuracy of pose annotation and reducing positional errors, and 2) an interactive learning system for 3D semantic segmentation with autonomous mobile robots, improving segmentation accuracy in new environments and predicting new object classes with minimal additional annotations. Both achievements focused on creating human-readable representations that facilitate a deeper understanding of service robots' learning processes.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The research is advancing smoothly, building upon the first year's development of a mixed reality-based interface that significantly reduced user burden. The second year focused on multimodal observations in extended reality (XR) for creating human-readable representations that facilitate a deeper understanding of service robots' learning processes. Experiments with collaborative tasks between humans and robots in XR have demonstrated enhanced interaction effectiveness, enabling more intuitive and direct user involvement in the learning process of the robots through XR.
|
| Strategy for Future Research Activity |
The final year will focus on the challenge of transforming complex latent spaces into intuitive representations within extended reality. The goal is to develop novel techniques that will allow users to visualize and interact with the latent space, thereby facilitating direct human intervention in the robot's learning process. The outcome is expected to enhance users' understanding and control over the knowledge formation in service robots.
|
| Causes of Carryover |
Acquire an expensive piece of research equipment in the final year.
|
