Fully Convolutional Encoder-Decoder With an Attention Mechanism for Practical Pedestrian Trajectory Prediction

• Published in: IEEE transactions on intelligent transportation systems Vol. 23; no. 11; pp. 20046 - 20060
• Main Authors: Chen, Kai, Song, Xiao, Yuan, Haitao, Ren, Xiaoxiang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; attention mechanism; Barriers; Coders; Convolution; Convolutional neural networks; Data processing; Datasets; Encoders-Decoders; Feature extraction; Force; Image annotation; Image segmentation; long short-term memory (LSTM); Markov processes; Pedestrian behavior; Pedestrians; Predictive models; Trajectory
• ISSN: 1524-9050; 1558-0016
• DOI: 10.1109/TITS.2022.3170874

Pedestrian trajectory prediction using video is essential for many practical traffic applications. Most existing pedestrian trajectory prediction methods are based on fully connected long short-term memory (LSTM) networks and perform well on public datasets. However, these methods still have three defects: a) Most of them rely on manual annotations to obtain information about the environment surrounding the subject pedestrian, which limits practical applications; b) The interaction among pedestrians and obstacles in a scene is little studied, which leads to greater prediction error; c) Traditional LSTM methods are based on the previous moment and ignore the correlation between the future and distant past states of the pedestrian, which generates unrealistic trajectories. To tackle these problems, first, in the stage of data processing, we use an image semantic segmentation algorithm to obtain multi-category obstacle information and design an end-to-end "Siamese Position Extraction" model to obtain more accurate pedestrian interaction data. Second, we design an end-to-end fully convolutional LSTM encoder-decoder with an attention mechanism (FLEAM) to overcome the shortcomings of LSTM. Third, we compare FLEAM with several state-of-the-art LSTM-based prediction methods on multiple video sequences in the datasets ETH, UCY and MOT20. The results show that our approach generates the same prediction error as the best results of the state-of-the-art method. However, FLEAM has more potential for practice application because it does not rely on manually annotated data. We further validate the effectiveness of FLEAM by employing manually annotated data, finding that it generates much less prediction error.