A synergistic use of chemometrics and deep learning improved the predictive performance of near-infrared spectroscopy models for dry matter prediction in mango fruit

• Published in: Chemometrics and intelligent laboratory systems Vol. 212; p. 104287
• Main Authors: Mishra, Puneet, Passos, Dário
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 15.05.2021
• Subjects: 1D-CNN; Artificial intelligence; Ensemble pre-processing; Fruit quality; Neural networks; 1D-CNN; Ensemble pre-processing; Neural networks; Artificial intelligence; Fruit quality
• ISSN: 0169-7439; 1873-3239
• DOI: 10.1016/j.chemolab.2021.104287

This study provides an innovative approach to improve deep learning (DL) models for spectral data processing with the use of chemometrics knowledge. The technique proposes pre-filtering the outliers using the Hotelling’s T2 and Q statistics obtained with partial least-square (PLS) analysis and spectral data augmentation in the variable domain to improve the predictive performance of DL models made on spectral data. The data augmentation is carried out by stacking the same data pre-processed with several pre-processing techniques such as standard normal variate, 1st derivatives, 2nd derivatives and their combinations. The performance of the approach is demonstrated on a real near-infrared (NIR) data set related to dry matter (DM) prediction in mango fruit. The data set consisted of a total 11,961 spectra and reference DM measurements. The results showed that removing the outliers and augmenting spectral data improved the predictive performance of DL models. Furthermore, this innovative approach not only improved DL models but attained the lowest root mean squared error of prediction (RMSEP) on the mango data set i.e., 0.79% compared to the best known RMSEP of 0.84%. Further, by removing outliers from the test set the RMSEP decreased to 0.75%. Several chemometrics approaches can complement DL models and should be widely explored in conjunction. •Advanced model to predict dry matter in mango is developed.•A synergistic use of chemometrics and deep learning (DL) is presented.•Outlier removal and pre-processing based data augmentation complemented DL.•A benchmark RMSEP of 0.79% was attained.