Supervised Feature Selection via Collaborative Neurodynamic Optimization

• Published in: IEEE transaction on neural networks and learning systems Vol. 35; no. 5; pp. 6878 - 6892
• Main Authors: Wang, Yadi, Wang, Wang, Pal, Nikhil R.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Biconvex optimization; Collaboration; collaborative neurodynamic optimization (CNO); Constraints; Feature extraction; Feature recognition; Feature selection; Machine learning; Mixed integer; mixed-integer optimization; Mutual information; Neural networks; Neurodynamics; Objective function; Optimization; Pattern recognition; Penalty function; Recurrent neural networks; Redundancy
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2022.3213167

As a crucial part of machine learning and pattern recognition, feature selection aims at selecting a subset of the most informative features from the set of all available features. In this article, supervised feature selection is at first formulated as a mixed-integer optimization problem with an objective function of weighted feature redundancy and relevancy subject to a cardinality constraint on the number of selected features. It is equivalently reformulated as a bound-constrained mixed-integer optimization problem by augmenting the objective function with a penalty function for realizing the cardinality constraint. With additional bilinear and linear equality constraints for realizing the integrality constraints, it is further reformulated as a bound-constrained biconvex optimization problem with two more penalty terms. Two collaborative neurodynamic optimization (CNO) approaches are proposed for solving the formulated and reformulated feature selection problems. One of the proposed CNO approaches uses a population of discrete-time recurrent neural networks (RNNs), and the other use a pair of continuous-time projection networks operating concurrently on two timescales. Experimental results on 13 benchmark datasets are elaborated to substantiate the superiority of the CNO approaches to several mainstream methods in terms of average classification accuracy with three commonly used classifiers.