An Improved Binary Quadratic Discriminant Analysis Classifier by Using Robust Regularization

• Published in: IEEE access Vol. 12; pp. 114951 - 114960
• Main Authors: Zaib, Alam, Khattak, Shahid, Mujtaba, Ghulam, Khan, Shahid, Al-Rasheed, Amal
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification algorithms; Covariance matrices; Data analysis; data classification; Discriminant analysis; Distributed databases; LDA; Least squares; Noise measurement; Parameter robustness; Performance degradation; Perturbation methods; QDA; Regularization; Robustness; Training data
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3437246

In many real classification problems where a limited number of training samples is available, the linear classifiers based on discriminant analysis are unable to deliver accurate results. Moreover, the testing and/or the training data can be erroneous due to noise contamination which further degrades their performance. Regularization techniques become imperative to deal with these problems. However, the existing regularization techniques, to some extent, mainly focus on data scarcity issues but completely ignore the noisy nature of the testing and/or the training data. We propose a novel regularized quadratic discriminant analysis (R-QDA) classifier which addresses both issues simultaneously. The procedure involves a reformulation of the discriminant function of the conventional QDA classifier into least square problems and then solving them by using regularized least squares (Reg-LS) based on <inline-formula> <tex-math notation="LaTeX">\ell _{2} </tex-math></inline-formula>-norm. In contrast to existing R-QDA techniques, the proposed R-QDA classifier employs two regularization parameters pertaining to each class, which can be independently selected by various robust techniques. Numerical results demonstrate the effectiveness of the proposed method over classical R-QDA methods, especially in high noise regimes.