Kullback-Leibler Reservoir Sampling for Fairness in Continual Learning

• Published in: 2024 IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN) pp. 460 - 466
• Main Authors: Nikoloutsopoulos, Sotirios, Koutsopoulos, Iordanis, Titsias, Michalis K.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 05.05.2024
• Subjects: Accuracy; Class Fairness; Continual Learning; Continuing education; Edge Learning; Machine learning; Replay-based methods; Reservoirs; Target tracking; Training; Training data
• DOI: 10.1109/ICMLCN59089.2024.10624806

Continual Learning (CL) aims to update a ML model with continuously arriving training data without having to retrain the model again from scratch. Continual Learning will play a critical role in future AI-enabled services at the network edge. We propose an online replay-based Continual Learning method for classification problems, in which the learner stores data points to a buffer and replays them during training. The core of our contribution is a new replay buffer content update policy that selects data points to store in the buffer by minimizing the Kullback-Leibler (KL) loss between the buffer distribution and a target distribution. This results in a new algorithm that generalizes previous replay-based methods such as Reservoir Sampling and Class-Balancing Reservoir Sampling. We set the target distribution to be dependent on the evolving empirical distribution of classes in the training data stream, and we parameterize it with a single parameter. This allows to model different target class distributions in the buffer such as the class distribution of the training data stream, the uniform class distribution, and a distribution with class percentages that are inversely proportional to those in the stream. We show that the KL-based approach improves the fairness in multi-class imbalanced data streams by resulting in more similar per-class test accuracy scores among all classes. We apply our method to several Continual Learning problems using the MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets, and we show that our method is better than the Reservoir Sampling and Class-Balancing Reservoir Sampling schemes in terms of class fairness and overall accuracy.