Personal object discovery in first-person videos

• Published in: IEEE transactions on image processing Vol. 24; no. 12; pp. 5789 - 5799
• Main Authors: Lu, Cewu, Liao, Renjie, Jia, Jiaya
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2015
• Subjects: Algorithms; Bayes methods; Bayes Theorem; Bayesian analysis; Cameras; Cluster Analysis; Construction; Dictionaries; Extraction; Feature extraction; first-person vision and wearable camera; Humans; Image color analysis; Image processing; Image Processing, Computer-Assisted - methods; Object detection; object discovery; Representations; scene understanding; Tasks; Telecommunications; Transaction processing; Video Recording - methods; Videos; first-person vision and wearable camera; object detection; scene understanding; Object discovery
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2015.2487868

People know and care for personal objects, which can be different for individuals. Automatically discovering personal objects is thus of great practical importance. We, in this paper, pursue this task with wearable cameras based on the common sense that personal objects generally company us in various scenes. With this clue, we exploit a new object-scene distribution for robust detection. Two technical challenges involved in estimating this distribution, i.e., scene extraction and unsupervised object discovery, are tackled. For scene extraction, we learn the latent representation instead of simply selecting a few frames from the videos. In object discovery, we build an interaction model to select frame-level objects and use nonparametric Bayesian clustering. Experiments verify the usefulness of our approach.