Online feature importance ranking based on sensitivity analysis

• Published in: Expert systems with applications Vol. 85; pp. 397 - 406
• Main Authors: Razmjoo, Alaleh, Xanthopoulos, Petros, Zheng, Qipeng Phil
• Format: Journal Article
• Language: English
• Published: New York Elsevier Ltd 01.11.2017; Elsevier BV
• Subjects: Classification; Data mining; Distance learning; Empirical analysis; Expert systems; Feature ranking; Filtration; Online learning; Preprocessing; Ranking; Sensitivity; Sensitivity analysis; Stochastic gradient descent; Sensitivity; Stochastic gradient descent; Feature ranking; Online learning
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2017.05.016

•A feature ranking method is developed based on an importance measure.•A feature's importance is calculated based on its impact on the class prediction.•The adaptations for correlated and dynamic feature spaces are presented.•The experimental results suggest better results compared with other methods. Online learning is a growing branch of data mining which allows all traditional data mining techniques to be applied on a online stream of data in real time. In this paper, we present a fast and efficient online sensitivity based feature ranking method (SFR) which is updated incrementally. We take advantage of the concept of global sensitivity and rank features based on their impact on the outcome of the classification model. In the feature selection part, we use a two-stage filtering method in order to first eliminate highly correlated and redundant features and then eliminate irrelevant features in the second stage. One important advantage of our algorithm is its generality, which means the method works for correlated feature spaces without preprocessing. It can be implemented along with any single-pass online classification method with separating hyperplane such as SVMs. The proposed method is primarily developed for online tasks, however, we achieve very significant experimental results in comparison with popular batch feature ranking/selection methods. We also perform experiments to compare the method with available online feature ranking methods. Empirical results suggest that our method can be successfully implemented in batch learning or online mode.