Predicting the future from first person (egocentric) vision: A survey

• Published in: Computer vision and image understanding Vol. 211; p. 103252
• Main Authors: Rodin, Ivan, Furnari, Antonino, Mavroeidis, Dimitrios, Farinella, Giovanni Maria
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.10.2021
• Subjects: Anticipation; Egocentric vision; First person vision; Future prediction; Egocentric vision; 41A10; 65D05; 65D17; First person vision; Anticipation; 41A05; Future prediction
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2021.103252

Egocentric videos can bring a lot of information about how humans perceive the world and interact with the environment, which can be beneficial for the analysis of human behaviour. The research in egocentric video analysis is developing rapidly thanks to the increasing availability of wearable devices and the opportunities offered by new large-scale egocentric datasets. As computer vision techniques continue to develop at an increasing pace, the tasks related to the prediction of future are starting to evolve from the need of understanding the present. Predicting future human activities, trajectories and interactions with objects is crucial in applications such as human–robot interaction, assistive wearable technologies for both industrial and daily living scenarios, entertainment and virtual or augmented reality. This survey summarizes the evolution of studies in the context of future prediction from egocentric vision making an overview of applications, devices, existing problems, commonly used datasets, models and input modalities. Our analysis highlights that methods for future prediction from egocentric vision can have a significant impact in a range of applications and that further research efforts should be devoted to the standardization of tasks and the proposal of datasets considering real-world scenarios such as the ones with an industrial vocation. •We review tasks, datasets and models for future predictions from egocentric vision.•Action anticipation, trajectory forecasting, next object prediction are the key tasks.•Multimodal training is important for predicting the future from egocentric vision.•Lack of benchmarks for building future prediction methods is the main problem.