Online Optimization Method of Learning Process for Meta-Learning

• Published in: Computer journal Vol. 67; no. 5; pp. 1645 - 1651
• Main Authors: Xu, Zhixiong, Zhang, Wei, Li, Ailin, Zhao, Feifei, Jing, Yuanyuan, Wan, Zheng, Cao, Lei, Chen, Xiliang
• Format: Journal Article
• Language: English
• Published: Oxford University Press 22.06.2024
• Subjects: deep reinforcement learning; gradient descent; Meta-learning; hyper-parameter
• ISSN: 0010-4620; 1460-2067
• DOI: 10.1093/comjnl/bxad089

Meta-learning is a pivotal and potentially influential machine learning approach to solve challenging problems in reinforcement learning. However, the costly hyper-parameter tuning for training stability of meta-learning is a known shortcoming and currently a hotspot of research. This paper addresses this shortcoming by introducing an online and easily trainable hyper-parameter optimization approach, called Meta Parameters Learning via Meta-Learning (MPML), to combine online hyper-parameter adjustment scheme into meta-learning algorithm, which reduces the need to tune hyper-parameters. Specifically, a basic learning rate for each training task is put forward. Besides, the proposed algorithm dynamically adapts multiple basic learning rate and a shared meta-learning rate through conducting gradient descent alongside the initial optimization steps. In addition, the sensitivity with respect to hyper-parameter choices in the proposed approach are also discussed compared with model-agnostic meta-learning method. The experimental results on reinforcement learning problems demonstrate MPML algorithm is easy to implement and delivers more highly competitive performance than existing meta-learning methods on a diverse set of challenging control tasks.