Synergetic application of an E-tongue, E-nose and E-eye combined with CNN models and an attention mechanism to detect the origin of black pepper

• Published in: Sensors and actuators. A. Physical. Vol. 357; p. 114417
• Main Authors: Wang, Shoucheng, Zhang, Qing, Liu, Chuanzheng, Wang, Zhiqiang, Gao, Jiyong, Yang, Xiaojing, Lan, Yubin
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.08.2023
• Subjects: Black pepper; Convolutional neural network; E-eye; E-nose; E-tongue; Information fusion; Black pepper; E-eye; Convolutional neural network; E-tongue; E-nose; Information fusion
• ISSN: 0924-4247; 1873-3069
• DOI: 10.1016/j.sna.2023.114417

As the most important and widely used spice in the world, black pepper is known as the “king of spices.” The geographical origin of black pepper greatly affects its quality and price. The existing physicochemical detection methods for distinguishing black pepper have inherent performance issues, such as expensive equipment, complex operations and high time consumption levels. This study proposes a novel method for identifying the origin of black pepper by synergically applying an E-tongue (ET), an E-nose (EN) and an E-eye (EE) in combination with a deep learning algorithm. First, taste and smell fingerprints were collected by ET and EN instruments, respectively, and the color, shape and texture information of different samples was collected by EE instruments. Three kinds of convolutional neural networks (CNNs) with one-dimensional or two-dimensional convolutional structures were designed and utilized to extract the feature information from the ET, EN and EE signals. Additionally, the Bayesian optimization algorithm (BOA) was applied to globally optimize the hyperparameters of the different CNN models. Then, a channel attention mechanism (CAM) module was introduced to achieve feature-level fusion for the three kinds of signals. Finally, a fully connected layer that uses a softmax algorithm was utilized for classifying the categories of black pepper. The experimental results showed that compared with employing a single sensory device, the proposed method yielded better recognition accuracy. Achieving accuracy, precision, recall and F1-score values of 99.71%, 0.997, 0.997 and 0.996 respectively, the proposed pattern recognition model obtained better classification results than the baseline models for the test set. This study introduces a rapid detection method for identifying the geographical origin of black pepper. [Display omitted] •A synergetic application of ET, EN and EE systems combined with deep learning is explored.•1D and 2D CNN models are designed to process different signal of intelligent senses.•A CAM is applied to fuse the features of the ET, EN and EE signals.•A low-cost and accurate detection technique is proposed.