| Project/Area Number |
22K17981
|
|---|
| Research Category |
Grant-in-Aid for Early-Career Scientists
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
Basic Section 61050:Intelligent robotics-related
|
|---|
| Research Institution | Ritsumeikan University |
|---|
Principal Investigator |
El Hafi Lotfi 立命館大学, 総合科学技術研究機構, 准教授 (90821554)
|
|---|
| Project Period (FY) |
2022-04-01 – 2025-03-31
|
| Project Status |
Completed (Fiscal Year 2024)
|
| Budget Amount *help |
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2024: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2023: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2022: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
|
|---|
| Keywords | Extended Reality / Human-Robot Interaction / Multimodal Learning / Knowledge Formation / Emergent Reality / emergent reality / knowledge formation / multimodal learning / human-robot interaction / extended reality |
|---|
| Outline of Research at the Start |
This proposal introduces the concept of Emergent Reality, a novel framework that combines multimodal unsupervised learning and human-robot interactions in extended reality to visualize the emergent phenomena derived from the robot's observations and intervene in its learning process.
|
| Outline of Final Research Achievements |
This research developed Emergent Reality (ER) for human-robot interactive multimodal learning in extended reality (XR). Throughout the research period, a system was designed and progressively enhanced to enable non-expert users to teach and collaborate with service robots. Initially, a mixed reality (MR) interface was created for teaching new objects and was shown through experiments to significantly reduce user burden compared to conventional speech-based interactions. The system was then advanced by developing an MR-based 6D-pose annotation tool to enhance robot manipulation, and an interactive learning system for 3D semantic segmentation to improve robot navigation in new environments. In the final stage, the full reality-virtuality continuum was explored by integrating augmented reality (AR), augmented virtuality (AV), and virtual reality (VR) for collaboration with service robots. The perception of these XR-based interactions was evaluated in a large-scale public demonstration.
|
| Academic Significance and Societal Importance of the Research Achievements |
This research laid the foundation toward Emergent Reality (ER), enabling human-robot interactive multimodal learning in extended reality (XR) using the full reality-virtuality continuum to lower the barrier for non-experts to teach robots, supporting the adoption of service robots in society.
|
