DIFA-GAN: Differential Attention-Guided Generative Adversarial Network for Unsupervised Video Forecasting

Unsupervised video forecasting, which aims to generate future frames by observing only the beginning few frames, has promising applications in autonomous driving, robot navigation, and video surveillance systems. However, realistic video forecasting is very challenging due to both complex content an...

Full description

Saved in:
Bibliographic Details
Published inIEEE signal processing letters Vol. 31; pp. 1795 - 1799
Main Author Jin, Beibei
Format Journal Article
LanguageEnglish
Published IEEE 2024
Subjects
Autonomous driving
Dams
Forecasting
Generative adversarial networks
Generators
Predictive models
Training
unsupervised learning
video forecasting
video understanding
Visualization
Online AccessGet full text

Cover

More Information
Summary:Unsupervised video forecasting, which aims to generate future frames by observing only the beginning few frames, has promising applications in autonomous driving, robot navigation, and video surveillance systems. However, realistic video forecasting is very challenging due to both complex content and diverse dynamic motions of video. We propose a Differential Attention-Guided Generative Adversarial Network (DIFA-GAN) for unsupervised video forecasting, where an attention mechanism is incorporated with a frame differential algorithm to exploit both spatial and temporal information. The model learns to focus selectively on parts related to motions and to offer different weights to them. Extensive experiments on two advanced datasets - KTH and KITTI show the advancement of our method.
ISSN:1070-9908
1558-2361
DOI:10.1109/LSP.2024.3400914