Insights of Feature Fusion for Video Memorability Prediction

• Published in: Digital TV and Wireless Multimedia Communication pp. 239 - 248
• Main Authors: Yue, Fumei, Li, Jing, Sun, Jiande
• Format: Book Chapter
• Language: English
• Published: Singapore Springer Singapore
• Series: Communications in Computer and Information Science
• Subjects: Feature fusion; Regression algorithms; Video memorability
• ISBN: 9811611939; 9789811611933
• ISSN: 1865-0929; 1865-0937
• DOI: 10.1007/978-981-16-1194-0_21

Researches on video memorability focus on predicting the probability of the content being remembered, which does make sense in both academic types of research and practical commercial. Therefore, we explore various feature fusion strategies on different regression models and give valuable suggestions for improving the prediction performance. To conclude persuasively, we test and analyze abundant performances derived from classical Random Forest (RF) and Support Vector Regression (SVR) algorithms with 7 state-of-the-art features and their combinations. Specifically, we first test the performances of the RF and SVR with the perspective of a single feature. And then we test the performance of each feature on RF and SVR by fusing semantic features, respectively. According to the single feature training results, the top-3 best features are selected to further evaluate the performance of their combinations. Then, we test the performance of the multi-feature fusion features on RF and SVR by adding semantic features. Finally, the comprehensive results indicate that the C3D feature fusing semantic features yield the best performance with the RF model for the long-term video memorability scores, and the LBP feature fusing semantic features yield the best performance with the SVR model for the short-term video memorability scores.