Predicate Logic Based Image Grammars for Complex Pattern Recognition

• Published in: International journal of computer vision Vol. 93; no. 2; pp. 141 - 161
• Main Authors: Shet, Vinay, Singh, Maneesh, Bahlmann, Claus, Ramesh, Visvanathan, Neumann, Jan, Davis, Larry
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.06.2011; Springer; Springer Nature B.V
• Subjects: Algorithms; Analysis; Artificial Intelligence; Automation; Cognition & reasoning; Computer Imaging; Computer Science; Computer vision; Detectors; Grammars; Hypotheses; Image Processing and Computer Vision; Knowledge; Logic; Machine vision; Materials handling; Optimization; Pattern Recognition; Pattern Recognition and Graphics; Predicate logic; Reasoning; Remote sensing; Sensors; Uncertainty; Vision; Object detection and classification; Human detection; Back propagation; Logical reasoning; Aerial image analysis; Stochastic image grammars; Bilattice
• ISSN: 0920-5691; 1573-1405
• DOI: 10.1007/s11263-010-0343-9

Predicate logic based reasoning approaches provide a means of formally specifying domain knowledge and manipulating symbolic information to explicitly reason about different concepts of interest. Extension of traditional binary predicate logics with the bilattice formalism permits the handling of uncertainty in reasoning, thereby facilitating their application to computer vision problems. In this paper, we propose using first order predicate logics, extended with a bilattice based uncertainty handling formalism, as a means of formally encoding pattern grammars, to parse a set of image features, and detect the presence of different patterns of interest. Detections from low level feature detectors are treated as logical facts and, in conjunction with logical rules, used to drive the reasoning. Positive and negative information from different sources, as well as uncertainties from detections, are integrated within the bilattice framework. We show that this approach can also generate proofs or justifications (in the form of parse trees) for each hypothesis it proposes thus permitting direct analysis of the final solution in linguistic form. Automated logical rule weight learning is an important aspect of the application of such systems in the computer vision domain. We propose a rule weight optimization method which casts the instantiated inference tree as a knowledge-based neural network, interprets rule uncertainties as link weights in the network, and applies a constrained, back-propagation algorithm to converge upon a set of rule weights that give optimal performance within the bilattice framework. Finally, we evaluate the proposed predicate logic based pattern grammar formulation via application to the problems of (a) detecting the presence of humans under partial occlusions and (b) detecting large complex man made structures as viewed in satellite imagery. We also evaluate the optimization approach on real as well as simulated data and show favorable results.