Visual Hindsight Self-Imitation Learning for Interactive Navigation

• Published in: IEEE access Vol. 12; pp. 83796 - 83809
• Main Authors: Kim, Kibeom, Lee, Moonhoen, Whoo Lee, Min, Shin, Kisung, Lee, Minsu, Zhang, Byoung-Tak
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Embedding; interactive navigation; Interactive systems; Labeling; Learning; Navigation; Reinforcement learning; Robot kinematics; Task analysis; Training; Trajectory; Vision; Visual hindsight; visual reinforcement learning; Visual tasks; Visualization
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3413864

Interactive visual navigation tasks, which involve following instructions to reach and interact with specific targets, are challenging not only because successful experiences are very rare but also because complex visual inputs require a substantial number of samples. Previous methods for these tasks often rely on intricately designed dense rewards or the use of expensive expert data for imitation learning. To tackle these challenges, we propose a novel approach, Visual Hindsight Self-Imitation Learning (VHS), which enables re-labeling in vision-based and partially observable environments through Prototypical Goal (PG) embedding. We introduce the PG embeddings, which are derived from experienced goal observations, as opposed to handling instructions as word embeddings. This embedding technique allows the agent to visually reinterpret its unsuccessful attempts, enabling vision-based goal re-labeling and self-imitation from enhanced successful experiences. Experimental results show that VHS outperforms existing techniques in interactive visual navigation tasks, confirming its superior performance, sample efficiency, and generalization.