Pooled shrinkage estimator for quadratic discriminant classifier: an analysis for small sample sizes in face recognition

• Published in: International journal of machine learning and cybernetics Vol. 9; no. 3; pp. 507 - 522
• Main Authors: Ali, Syed Shahnewaz, Howlader, Tamanna, Rahman, S. M. Mahbubur
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2018; Springer Nature B.V
• Subjects: Artificial Intelligence; Biometric identification; Biometrics; Classification; Classifiers; Complex Systems; Computational Intelligence; Control; Covariance matrix; Discriminant analysis; Engineering; Face recognition; Feature extraction; Mechatronics; Monte Carlo simulation; Optimization techniques; Original Article; Pattern Recognition; Robotics; Sample size; Shrinkage; Support vector machines; Systems Biology; Monte Carlo simulations; Quadratic discriminant classifier; Face recognition; High dimensional covariance estimation; Shrinkage estimator
• ISSN: 1868-8071; 1868-808X
• DOI: 10.1007/s13042-016-0549-4

The quadratic discriminant classifier (QDC) is a well-known parametric Bayesian classifier that has been successfully applied to statistical pattern recognition problems. One such application is in automatic face recognition where the number of training images per subject is often found to be much less than the length of the facial features. In such a case, the QDC cannot be used because the class-specific covariance matrix on which it depends is either poorly estimated or singular thereby resulting in unacceptable classifier performance. High dimensional covariance estimation techniques such as shrinkage can alleviate this problem but only to a certain extent. This paper presents a computationally simple yet effective solution for further improving the QDC performance in small sample size scenarios. The proposed technique adopts a strategy of combining the class-specific shrinkage estimates of the covariance matrix to obtain a pooled shrinkage estimate, which is then plugged into the QDC. Experiments indicate that the proposed classifier leads to remarkable improvement in face recognition accuracy as compared to the existing classifiers such as the nearest neighbor, support vector machine and naive Bayes, irrespective of the nature of the database and feature extraction method. Monte Carlo simulations reveal that this improvement is due to the much lower mean squared error of the pooled shrinkage estimator which offers greater stability to the QDC.