Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 29; no. 6; pp. 1005 - 1018
• Main Authors: Tae-Kyun Kim, Kittler, J., Cipolla, R.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.06.2007; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Cameras; canonical correlation; canonical correlation analysis; Cluster Analysis; Computer science; control theory; systems; Computer vision; Correlation; Correlation analysis; Discriminant Analysis; Exact sciences and technology; Face recognition; Illumination; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image recognition; image sets; Information Storage and Retrieval - methods; Learning; Lighting; Linear discriminant analysis; Matching; Methods; Neural networks; Numerical Analysis, Computer-Assisted; Object recognition; orthogonal subspace method; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; principal angles; Recognition; Reproducibility of Results; Robustness; Sensitivity and Specificity; Signal Processing, Computer-Assisted; Statistics as Topic; Studies; Subtraction Technique; Vectors; Image recognition; Image processing; Canonical correlation; Discriminant function; Correlation method; Parametric method; Canonical analysis; Image matching; Vector space; Lighting; Classification; Facies; principal angles; Database; Illumination; orthogonal subspace method; Robustness; linear discriminant analysis; Pattern analysis; Computer vision; Discriminant analysis; Motion estimation; Face recognition; Subspace method; Pattern recognition; canonical correlation analysis; Object recognition; Luminance; Correlation analysis; Artificial intelligence; image sets
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2007.1037

We address the problem of comparing sets of images for object recognition, where the sets may represent variations in an object's appearance due to changing camera pose and lighting conditions. canonical correlations (also known as principal or canonical angles), which can be thought of as the angles between two d-dimensional subspaces, have recently attracted attention for image set matching. Canonical correlations offer many benefits in accuracy, efficiency, and robustness compared to the two main classical methods: parametric distribution-based and nonparametric sample-based matching of sets. Here, this is first demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations. Motivated by their proven effectiveness, a novel discriminative learning method over sets is proposed for set classification. Specifically, inspired by classical linear discriminant analysis (LDA), we develop a linear discriminant function that maximizes the canonical correlations of within-class sets and minimizes the canonical correlations of between-class sets. Image sets transformed by the discriminant function are then compared by the canonical correlations. Classical orthogonal subspace method (OSM) is also investigated for the similar purpose and compared with the proposed method. The proposed method is evaluated on various object recognition problems using face image sets with arbitrary motion captured under different illuminations and image sets of 500 general objects taken at different views. The method is also applied to object category recognition using ETH-80 database. The proposed method is shown to outperform the state-of-the-art methods in terms of accuracy and efficiency