Deep Learning in Next-Frame Prediction: A Benchmark Review

• Published in: IEEE access Vol. 8; p. 1
• Main Authors: Zhou, Yufan, Dong, Haiwei, El Saddik, Abdulmotaleb
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Computer architecture; Computer vision; Decision making; Deep learning; Feature extraction; Frame prediction architecture; Gallium nitride; Generative adversarial networks; loss function design; Machine learning; Predictions; Predictive models; state-of-the-art evaluation; Task analysis
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2020.2987281

As an unsupervised representation problem in deep learning, next-frame prediction is a new, promising direction of research in computer vision, predicting possible future images by presenting historical image information. It provides extensive application value in robot decision making and autonomous driving. In this paper, we introduce recent state-of-the-art next-frame prediction networks and categorize them into two architectures: sequence-to-one architecture and sequence-to-sequence architecture. After comparing these approaches by analyzing the loss function design and network architecture, the pros and cons are analyzed. Based on the off-the-shelf data-sets and the corresponding evaluation metrics, the performance of the aforementioned approaches is quantitatively compared. The future promising research directions are pointed out at last.