Class-Incremental Learning of Convolutional Neural Networks Based on Double Consolidation Mechanism

• Published in: IEEE access Vol. 8; pp. 172553 - 172562
• Main Authors: Jin, Leilei, Liang, Hong, Yang, Changsheng
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Biological neural networks; Class-incremental learning; Classification; Classification algorithms; Consolidation; Convolutional neural networks; Data transmission; Distillation; knowledge distillation; Knowledge engineering; Machine learning; Process parameters; Task analysis; Training; visual recognition; weight consolidation
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2020.3025558

Class-incremental learning is a model learning technique that can help classification models incrementally learn about new target classes and realize knowledge accumulation. It has become one of the major concerns of the machine learning and classification community. To overcome the catastrophic forgetting that occurs when the network is trained sequentially on a multi-class data stream, a double consolidation class-incremental learning (DCCIL) method is proposed. In the incremental learning process, the network parameters are adjusted by combining knowledge distillation and elastic weight consolidation, so that the network can better maintain the recognition ability of the old classes while learning the new ones. The incremental learning experiment is designed, and the proposed method is compared with the popular incremental learning methods such as EWC, LwF, and iCaRL. Experimental results show that the proposed DCCIL method can achieve better incremental accuracy than that of the current popular incremental learning algorithms, which can effectively improve the expansibility and intelligence of the classification model.