Learning Regularized LDA by Clustering

• Published in: IEEE transaction on neural networks and learning systems Vol. 25; no. 12; pp. 2191 - 2201
• Main Authors: Pang, Yanwei, Wang, Shuang, Yuan, Yuan
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2014; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Clustering; Computation; Dimensionality reduction; Discriminant analysis; Face; Face recognition; feature extraction; Learning; linear discriminant analysis (LDA); Neural networks; Scatter; Silicon; Training; Vectors; Dimensionality reduction; face recognition; linear discriminant analysis (LDA); feature extraction
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2014.2306844

As a supervised dimensionality reduction technique, linear discriminant analysis has a serious overfitting problem when the number of training samples per class is small. The main reason is that the between- and within-class scatter matrices computed from the limited number of training samples deviate greatly from the underlying ones. To overcome the problem without increasing the number of training samples, we propose making use of the structure of the given training data to regularize the between- and within-class scatter matrices by between- and within-cluster scatter matrices, respectively, and simultaneously. The within- and between-cluster matrices are computed from unsupervised clustered data. The within-cluster scatter matrix contributes to encoding the possible variations in intraclasses and the between-cluster scatter matrix is useful for separating extra classes. The contributions are inversely proportional to the number of training samples per class. The advantages of the proposed method become more remarkable as the number of training samples per class decreases. Experimental results on the AR and Feret face databases demonstrate the effectiveness of the proposed method.