Relief-based feature selection: Introduction and review

• Published in: Journal of biomedical informatics Vol. 85; pp. 189 - 203
• Main Authors: Urbanowicz, Ryan J., Meeker, Melissa, La Cava, William, Olson, Randal S., Moore, Jason H.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.09.2018
• Subjects: Algorithms; Computational Biology - methods; Data Mining - methods; Epistasis; Feature interaction; Feature selection; Feature weighting; Filter; Humans; Models, Statistical; Regression Analysis; ReliefF; Software; Feature selection; Epistasis; Filter; Feature interaction; ReliefF; Feature weighting
• ISSN: 1532-0464; 1532-0480; 1532-0480
• DOI: 10.1016/j.jbi.2018.07.014

[Display omitted] •Relief-based feature selection methods (RBAs) are reviewed in detailed context.•RBAs can detect interactions without examining pairwise combinations.•Iterative RBAs have been developed to scale them up to very large feature spaces.•Research focused on core algorithms, iterative scaling, and data type flexibility. Feature selection plays a critical role in biomedical data mining, driven by increasing feature dimensionality in target problems and growing interest in advanced but computationally expensive methodologies able to model complex associations. Specifically, there is a need for feature selection methods that are computationally efficient, yet sensitive to complex patterns of association, e.g. interactions, so that informative features are not mistakenly eliminated prior to downstream modeling. This paper focuses on Relief-based algorithms (RBAs), a unique family of filter-style feature selection algorithms that have gained appeal by striking an effective balance between these objectives while flexibly adapting to various data characteristics, e.g. classification vs. regression. First, this work broadly examines types of feature selection and defines RBAs within that context. Next, we introduce the original Relief algorithm and associated concepts, emphasizing the intuition behind how it works, how feature weights generated by the algorithm can be interpreted, and why it is sensitive to feature interactions without evaluating combinations of features. Lastly, we include an expansive review of RBA methodological research beyond Relief and its popular descendant, ReliefF. In particular, we characterize branches of RBA research, and provide comparative summaries of RBA algorithms including contributions, strategies, functionality, time complexity, adaptation to key data characteristics, and software availability.