Random Forests with ensemble of feature spaces

• Published in: Pattern recognition Vol. 47; no. 10; pp. 3429 - 3437
• Main Authors: Zhang, Le, Suganthan, Ponnuthurai Nagaratnam
• Format: Journal Article
• Language: English
• Published: Kidlington Elsevier Ltd 01.10.2014; Elsevier
• Subjects: Applied sciences; Benchmarking; Classification; Diversity; Ensemble; Exact sciences and technology; Forests; Information theory; Information, signal and communications theory; Pattern recognition; Post-processing; Roots; Rotation; Telecommunications and information theory; Transformation; Trees; Transformation; Rotation; Ensemble; Classification; Diversity; Learning; Performance evaluation; Accuracy; Data processing; Localization
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2014.04.001

Random Forests receive much attention from researchers because of their excellent performance. As Breiman suggested, the performance of Random Forests depends on the strength of the weak learners in the forests and the diversity among them. However, in the literature, many researchers only considered pre-processing of the data or post-processing of the Random Forests models. In this paper, we propose a new method to increase the diversity of each tree in the forests and thereby improve the overall accuracy. During the training process of each individual tree in the forest, different rotation spaces are concatenated into a higher space at the root node. Then the best split is exhaustively searched within this higher space. The location where the best split lies decides which rotation method to be used for all subsequent nodes. The performance of the proposed method here is evaluated on 42 benchmark data sets from various research fields and compared with the standard Random Forests. The results show that the proposed method improves the performance of the Random Forests in most cases. •We propose two rotation/transformation based RF classifiers firstly.•We generate an ensemble classifier with relatively high diversity .•All the methods have the similar properties with standard RF.•All the proposed methods outperform standard Random Forests.•The RF with ensemble of spaces outperforms others in a large margin.