Incorporating Deep Learning and Error Potential Feedback to a BMI System to Enhance User Experience

2015 Fiscal Year Annual Research Report

• Project/Area Number: 15H06922
• Research Institution: Advanced Telecommunications Research Institute International
• Principal Investigator: Penaloza C. 株式会社国際電気通信基礎技術研究所, その他部局等, 研究員 (80753532)
• Project Period (FY): 2015-08-28 – 2017-03-31
• Keywords: Android / Deep Learning / Brain Machine Interface / Error Potential / Robot Learning

Outline of Annual Research Achievements

In this research we are proposing a novel BMI system that uses Deep Learning algorithms to achieve higher accuracy rates, and allows a user to tele-operate a robot that will gradually learn user commands. By detecting users’ error potentials (ErrP) that spontaneously appear during the observation of a mistake made by the robot, the robot will be advised of its own mistakes and improve its performance. Providing learning and error-perception capabilities to the BMI-robot system will allow it to achieve certain degree of automation and the operator will be relieved from constant concentration thus reducing mental fatigue.
Until now, we developed a deep learning algorithm that is able to recognize EEG signal based Error potentials with 80% precision using opensource datasets. We then conducted an experiment to collect Error related EEG data from human participants and the offline analysis using the developed algorithm is under development.
An Android robot teleoperating framework using BMI was also developed and currently it is been updated to provide participants with realistic visual and haptic feedback. After finishing setting up the framework, we will integrate the hardware and software components in order to test the system in an additional experiment.

Current Status of Research Progress

2: Research has progressed on the whole more than it was originally planned.

Reason

Although the order of development has changed from the initial proposal, in overall, progress has been made in the project and we look forward to reach the proposed goal within the planned schedule.

Strategy for Future Research Activity

The next activities are as follows:
-Finish setting up the hardware configuration of the system for the experiment.
- develop a robot learning framework for a particular task to be used in the experiment.
- Integrate the hardware and software components of the system.
- Perform a final experiment to corroborate the research hypothesis.