DiffusionLSTM: A Framework for Image Sequence Generation and Its Application to Oil Spill Monitoring and Prediction

• Published in: IEEE transactions on geoscience and remote sensing Vol. 62; pp. 1 - 13
• Main Authors: Lyu, Xinrong, Han, Hongbo, Ren, Peng, Grecos, Christos
• Format: Journal Article
• Language: English
• Published: New York IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Data models; Deep learning; Denoising diffusion probabilistic model (DDPM); Emergency management; Emergency preparedness; Energy sources; Environmental monitoring; Image enhancement; Information processing; Long short-term memory; long short-term memory (LSTM); Marine environment; Mathematical models; Monitoring; Numerical models; oil spill monitoring; Oil spills; Oils; OpenOil; Prediction models; Predictive models; Remote monitoring; Remote sensing; Satellite imagery; Tanker ships; Trajectory
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2024.3425539

Oil remains the most important energy source in the world today, and tankers are its main modes of transportation. However, there is a high risk of oil spills, which can cause serious damage to the ecological environment. Remote sensing monitoring is one of the most common processes of emergency management for oil spills. Short interval and uninterrupted remote sensing sequence data are crucial for monitoring and tracing oil spill events. However, the long revisit period of satellites poses a challenge of data scarcity for oil spill monitoring. To address the issue of insufficient remote sensing image data in oil spill monitoring and prediction, we propose a joint modeling approach that combines deep learning and numerical models to enhance the monitoring efficiency. First, a DiffusionLSTM network is proposed based on the diffusion model's ability to generate images and the capability of long short-term memory (LSTM) network to extract temporal information. The proposed network can learn the evolution pattern of images from historical remote sensing data and predict future scenarios. Comparative experiments on the MODSD dataset show that our proposed model achieved a significant improvement compared to traditional time-series image prediction models (ConvLSTM and GAN-LSTM). Second, a trajectory model based on numerical simulation methods is established using OpenOil. Taking into account the differences in different marine areas, we accurately reconstruct oil spill trajectories by calibrating the wind drift coefficients. For Sanchi oil spill incident, the error is reduced approximately to 2500 m. Finally, through fusing the images generated by DiffusionLSTM and the oil spill trajectories predicted by OpenOil, short time interval oil spill scene images have been generated efficiently to improve the monitoring efficiency.