User-Centric Learning and Evaluation of Interactive Segmentation Systems

• Published in: International journal of computer vision Vol. 100; no. 3; pp. 261 - 274
• Main Authors: Kohli, Pushmeet, Nickisch, Hannes, Rother, Carsten, Rhemann, Christoph
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.12.2012; Springer; Springer Nature B.V
• Subjects: Accuracy; Algorithms; Applied sciences; Artificial Intelligence; Computer Imaging; Computer Science; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Control theory. Systems; Datasets; End users; Exact sciences and technology; Image Processing and Computer Vision; Interactive computer systems; International; Machine learning; Machine vision; Pattern Recognition; Pattern Recognition and Graphics; Pattern recognition. Digital image processing. Computational geometry; Robotics; Software; Studies; Technology application; User behavior; Vision; Vision systems; United Kingdom > UK; Learning; Interactive systems; Image segmentation; Computer vision; Parameter estimation; Image processing; Robotics; Interactive system; User interface; Interactive video; Vector support machine; Learning algorithm; Artificial intelligence; User behavior
• ISSN: 0920-5691; 1573-1405
• DOI: 10.1007/s11263-012-0537-4

Many successful applications of computer vision to image or video manipulation are interactive by nature. However, parameters of such systems are often trained neglecting the user. Traditionally, interactive systems have been treated in the same manner as their fully automatic counterparts. Their performance is evaluated by computing the accuracy of their solutions under some fixed set of user interactions. In this paper, we study the problem of evaluating and learning interactive segmentation systems which are extensively used in the real world. The key questions in this context are how to measure (1) the effort associated with a user interaction, and (2) the quality of the segmentation result as perceived by the user. We conduct a user study to analyze user behavior and answer these questions. Using the insights obtained from these experiments, we propose a framework to evaluate and learn interactive segmentation systems which brings the user in the loop. The framework is based on the use of an active robot user—a simulated model of a human user. We show how this approach can be used to evaluate and learn parameters of state-of-the-art interactive segmentation systems. We also show how simulated user models can be integrated into the popular max-margin method for parameter learning and propose an algorithm to solve the resulting optimisation problem.