Human action recognition using fusion of multiview and deep features: an application to video surveillance

• Published in: Multimedia tools and applications Vol. 83; no. 5; pp. 14885 - 14911
• Main Authors: Khan, Muhammad Attique, Javed, Kashif, Khan, Sajid Ali, Saba, Tanzila, Habib, Usman, Khan, Junaid Ali, Abbasi, Aaqif Afzaal
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2024; Springer Nature B.V
• Subjects: Artificial intelligence; Artificial neural networks; Computer Communication Networks; Computer Science; Correlation coefficients; Data Structures and Information Theory; Human activity recognition; Multimedia Information Systems; Parameters; Special Purpose and Application-Based Systems; Surveillance; Human action recognition; Deep features; Multiview features; Features fusion; Recognition
• DOI: 10.1007/s11042-020-08806-9
• ISSN: 1573-7721; 1380-7501

Human Action Recognition (HAR) has become one of the most active research area in the domain of artificial intelligence, due to various applications such as video surveillance. The wide range of variations among human actions in daily life makes the recognition process more difficult. In this article, a new fully automated scheme is proposed for Human action recognition by fusion of deep neural network (DNN) and multiview features. The DNN features are initially extracted by employing a pre-trained CNN model name VGG19. Subsequently, multiview features are computed from horizontal and vertical gradients, along with vertical directional features. Afterwards, all features are combined in order to select the best features. The best features are selected by employing three parameters i.e. relative entropy, mutual information, and strong correlation coefficient (SCC). Furthermore, these parameters are used for selection of best subset of features through a higher probability based threshold function. The final selected features are provided to Naive Bayes classifier for final recognition. The proposed scheme is tested on five datasets name HMDB51, UCF Sports, YouTube, IXMAS, and KTH and the achieved accuracy were 93.7%, 98%, 99.4%, 95.2%, and 97%, respectively. Lastly, the proposed method in this article is compared with existing techniques. The resuls shows that the proposed scheme outperforms the state of the art methods.