Autonomous grasping of 3-D objects by a vision-actuated robot arm using Brain–Computer Interface

• Published in: Biomedical signal processing and control Vol. 84; p. 104765
• Main Authors: Rakshit, Arnab, Pramanick, Shraman, Bagchi, Anurag, Bhattacharyya, Saugat
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.07.2023
• Subjects: Brain–Computer Interface; Convolutional Neural Network; Electroencephalogram; Overlapping Object GraspNet; Robot; Convolutional Neural Network; Electroencephalogram; Brain–Computer Interface; Robot; Overlapping Object GraspNet
• ISSN: 1746-8094; 1746-8108
• DOI: 10.1016/j.bspc.2023.104765

A major drawback of a Brain–Computer Interface-based robotic manipulation is the complex trajectory planning of the robot arm to be carried out by the user for reaching and grasping an object. The present paper proposes an intelligent solution to the existing problem by incorporating a novel Convolutional Neural Network (CNN)-based grasp detection network that enables the robot to reach and grasp the desired object (including overlapping objects) autonomously using a RGB-D camera. This network uses a simultaneous object and grasp detection to affiliate each estimated grasp with its corresponding object. The subject uses motor imagery brain signals to control the pan and tilt angle of a RGB-D camera mounted on a robot link to bring the desired object inside its Field-of-view presented through a display screen while the objects appearing on the screen are selected using the P300 brain pattern. The robot uses inverse kinematics along with the RGB-D camera information to autonomously reach the selected object and the object is grasped using proposed grasping strategy. The overall BCI system outperforms other comparative systems involving manual trajectory planning significantly. The overall accuracy, steady-state error, and settling time of the proposed system are 93.4%, 0.05%, and 15.92 s, respectively. The system also shows a significant reduction of the workload of the operating subjects in comparison to manual trajectory planning based approaches for reaching and grasping. •A novel BCI scheme to localize, select and grasp the desired object in a multi-object scene.•Shared cognitive control based approach.•Accurate grasping with minimal cognitive effort.•A CNN based novel robotic grasp detection network(OOGNet) to grasp the desired object partially overlapped by other objects.•Subjects relieved from complex robot trajectory planning.•Requires very little subject training.•Significant reduction of overall workload of the subject.•Significant improvement of success rate.•Minimizes the settling-time and positional steady-state error of the system.