ROSE: robust online self-adjusting ensemble for continual learning on imbalanced drifting data streams

• Published in: Machine learning Vol. 111; no. 7; pp. 2561 - 2599
• Main Authors: Cano, Alberto, Krawczyk, Bartosz
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.07.2022; Springer Nature B.V
• Subjects: Adaptation; Algorithms; Artificial Intelligence; Classifiers; Computer Science; Control; Data transmission; Distance learning; Drift; Machine Learning; Mechatronics; Natural Language Processing (NLP); Online instruction; Robotics; Robustness; Simulation and Modeling; Statistical analysis; Imbalanced data; Data streams; Continual learning; Concept drift; Online learning
• ISSN: 0885-6125; 1573-0565
• DOI: 10.1007/s10994-022-06168-x

Data streams are potentially unbounded sequences of instances arriving over time to a classifier. Designing algorithms that are capable of dealing with massive, rapidly arriving information is one of the most dynamically developing areas of machine learning. Such learners must be able to deal with a phenomenon known as concept drift, where the data stream may be subject to various changes in its characteristics over time. Furthermore, distributions of classes may evolve over time, leading to a highly difficult non-stationary class imbalance. In this work we introduce Robust Online Self-Adjusting Ensemble (ROSE), a novel online ensemble classifier capable of dealing with all of the mentioned challenges. The main features of ROSE are: (1) online training of base classifiers on variable size random subsets of features; (2) online detection of concept drift and creation of a background ensemble for faster adaptation to changes; (3) sliding window per class to create skew-insensitive classifiers regardless of the current imbalance ratio; and (4) self-adjusting bagging to enhance the exposure of difficult instances from minority classes. The interplay among these features leads to an improved performance in various data stream mining benchmarks. An extensive experimental study comparing with 30 ensemble classifiers shows that ROSE is a robust and well-rounded classifier for drifting imbalanced data streams, especially under the presence of noise and class imbalance drift, while maintaining competitive time complexity and memory consumption. Results are supported by a thorough non-parametric statistical analysis.