Deep Reinforcement Learning for Multiobjective Optimization

• Published in: IEEE transactions on cybernetics Vol. 51; no. 6; pp. 3103 - 3114
• Main Authors: Li, Kaiwen, Zhang, Tao, Wang, Rui
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Decomposition; Deep learning; Deep reinforcement learning (DRL); Iterative methods; Machine learning; Mathematical models; Modeling; Mopping; multiobjective optimization; Multiple objective analysis; Neural networks; Optimization; Parameters; Pareto optimization; Pointer Network; Reinforcement learning; Training; Traveling salesman problem; Traveling salesman problems; Urban areas
• DOI: 10.1109/TCYB.2020.2977661
• ISSN: 2168-2267; 2168-2275

This article proposes an end-to-end framework for solving multiobjective optimization problems (MOPs) using deep reinforcement learning (DRL), that we call DRL-based multiobjective optimization algorithm (DRL-MOA). The idea of decomposition is adopted to decompose the MOP into a set of scalar optimization subproblems. Then, each subproblem is modeled as a neural network. Model parameters of all the subproblems are optimized collaboratively according to a neighborhood-based parameter-transfer strategy and the DRL training algorithm. Pareto-optimal solutions can be directly obtained through the trained neural-network models. Specifically, the multiobjective traveling salesman problem (MOTSP) is solved in this article using the DRL-MOA method by modeling the subproblem as a Pointer Network. Extensive experiments have been conducted to study the DRL-MOA and various benchmark methods are compared with it. It is found that once the trained model is available, it can scale to newly encountered problems with no need for retraining the model. The solutions can be directly obtained by a simple forward calculation of the neural network; thereby, no iteration is required and the MOP can be always solved in a reasonable time. The proposed method provides a new way of solving the MOP by means of DRL. It has shown a set of new characteristics, for example, strong generalization ability and fast solving speed in comparison with the existing methods for multiobjective optimizations. The experimental results show the effectiveness and competitiveness of the proposed method in terms of model performance and running time.