A principled approach to detecting surprising events in video

• Published in: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) Vol. 1; pp. 631 - 637 vol. 1
• Main Authors: Itti, L., Baldi, P.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2005
• Subjects: Bioinformatics; Brain modeling; Computer science; Computer vision; Event detection; Genomics; Humans; Image analysis; Neuroscience; Streaming media
• DOI: 10.1109/CVPR.2005.40
• ISBN: 0769523722; 9780769523729
• ISSN: 1063-6919

Primates demonstrate unparalleled ability at rapidly orienting towards important events in complex dynamic environments. During rapid guidance of attention and gaze towards potential objects of interest or threats, often there is no time for detailed visual analysis. Thus, heuristic computations are necessary to locate the most interesting events in quasi real-time. We present a new theory of sensory surprise, which provides a principled and computable shortcut to important information. We develop a model that computes instantaneous low-level surprise at every location in video streams. The algorithm significantly correlates with eye movements of two humans watching complex video clips, including television programs (17,936 frames, 2,152 saccadic gaze shifts). The system allows more sophisticated and time-consuming image analysis to be efficiently focused onto the most surprising subsets of the incoming data.