Improved binary particle swarm optimization for feature selection with new initialization and search space reduction strategies

• Published in: Applied soft computing Vol. 106; p. 107302
• Main Authors: Li, An-Da, Xue, Bing, Zhang, Mengjie
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.07.2021
• Subjects: Classification; Feature selection; Initialization; Particle swarm optimization; Initialization; Feature selection; Particle swarm optimization; Classification
• DOI: 10.1016/j.asoc.2021.107302
• ISSN: 1568-4946

Feature selection (FS) is an important preprocessing technique for dimensionality reduction in classification problems. Particle swarm optimization (PSO) algorithms have been widely used as the optimizers for FS problems. However, with the increase of data dimensionality, the search space expands dramatically, which proposes significant challenges for optimization methods, including PSO. In this paper, we propose an improved sticky binary PSO (ISBPSO) algorithm for FS. ISBPSO adopts three new mechanisms based on a recently proposed binary PSO variant, sticky binary particle swarm optimization (SBPSO), to improve the evolutionary performance. First, a new initialization strategy using the feature weighting information based on mutual information is proposed. Second, a dynamic bits masking strategy for gradually reducing the search space during the evolutionary process is proposed. Third, based on the framework of memetic algorithms, a refinement procedure conducting genetic operations on the personal best positions of ISBPSO is used to alleviate the premature convergence problem. The results on 12 UCI datasets show that ISBPSO outperforms six benchmark PSO-based FS methods and two conventional FS methods (sequential forward selection and sequential backward selection) — ISBPSO obtains either higher or similar accuracies with fewer features in most cases. Moreover, ISBPSO substantially reduces the computation time compared with benchmark PSO-based FS methods. Further analysis shows that all the three proposed mechanisms are effective for improving the search performance of ISBPSO. •An improved binary PSO algorithm is proposed for feature selection.•A new initialization strategy using the feature weighting results is proposed.•A bits masking strategy for dynamically reducing the search space is proposed.•The genetic operations are adopted to alleviate the premature convergence problem.•The proposed algorithm is effective and efficient for feature selection.