Multitask Feature Learning as Multiobjective Optimization: A New Genetic Programming Approach to Image Classification

• Published in: IEEE transactions on cybernetics Vol. 53; no. 5; pp. 3007 - 3020
• Main Authors: Bi, Ying, Xue, Bing, Zhang, Mengjie
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification; Evolutionary computation (EC); Feature extraction; feature learning; Genetic algorithms; Genetic programming; genetic programming (GP); Image classification; Machine learning; multiobjective optimization; Multiple objective analysis; multitask learning; Optimization; Representation learning; Representations; Solution space; Task analysis; Training; Training data
• ISSN: 2168-2267; 2168-2275
• DOI: 10.1109/TCYB.2022.3174519

Feature learning is a promising approach to image classification. However, it is difficult due to high image variations. When the training data are small, it becomes even more challenging, due to the risk of overfitting. Multitask feature learning has shown the potential for improving generalization. However, existing methods are not effective for handling the case that multiple tasks are partially conflicting. Therefore, for the first time, this article proposes to solve a multitask feature learning problem as a multiobjective optimization problem by developing a genetic programming approach with a new representation to image classification. In the new approach, all the tasks share the same solution space and each solution is evaluated on multiple tasks so that the objectives of all the tasks can be optimized simultaneously using a single population. To learn effective features, a new and compact program representation is developed to allow the new approach to evolving solutions shared across tasks. The new approach can automatically find a diverse set of nondominated solutions that achieve good tradeoffs between different tasks. To further reduce the risk of overfitting, an ensemble is created by selecting nondominated solutions to solve each image classification task. The results show that the new approach significantly outperforms a large number of benchmark methods on six problems consisting of 15 image classification datasets of varying difficulty. Further analysis shows that these new designs are effective for improving the performance. The detailed analysis clearly reveals the benefits of solving multitask feature learning as multiobjective optimization in improving the generalization.