Human object interaction detection based on feature optimization and key human-object enhancement

• Published in: Journal of visual communication and image representation Vol. 93; p. 103824
• Main Authors: Ye, Qing, Wang, Xikun, Li, Rui, Zhang, Yongmei
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.05.2023
• Subjects: FOFR-CNN; Graph convolutional network; Human object interaction detection; Key human-object enhancement; Human object interaction detection; FOFR-CNN; Graph convolutional network; Key human-object enhancement
• ISSN: 1047-3203; 1095-9076
• DOI: 10.1016/j.jvcir.2023.103824

Aiming at the problem of unclear or missing human object interaction behavior objects in complex background, we propose a human object interaction detection algorithm based on feature optimization and key human-object enhancement. In order to solve the problem of missing human behavior objects, we propose Feature Optimized Faster Region Convolutional Neural Network (FOFR-CNN). FOFR-CNN is an object detection network optimized by multi-scale feature optimization algorithm, taking into account both image semantics and image structure. In order to reduce the interference of complex background, we propose a Key Human-Object Enhancement Network. The network uses an instance-based method to enhance the features of interactive objects. In order to enrich the interaction information, we use the graph convolutional network. Experimental results on HICO-DET, V-COCO and HOI-A datasets show that the proposed algorithm has significantly improved accuracy and multi-scale object detection ability compared with other human object interaction algorithms.