Convolutional Recurrent Reconstructive Network for Spatiotemporal Anomaly Detection in Solder Paste Inspection

• Published in: IEEE transactions on cybernetics Vol. 52; no. 6; pp. 4688 - 4700
• Main Authors: Yoo, Yong-Ho, Kim, Ue-Hwan, Kim, Jong-Hwan
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Anomalies; Anomaly detection; Attention mechanism; Circuit boards; Coders; convolutional LSTM (ConvLSTM); Decoding; Decomposition; Defective products; Defects; Encoders-Decoders; Image reconstruction; Inspection; Malfunctions; one-class anomaly detection; Printers; Printers (data processing); recurrent auto-encoder (RAE); Soldering; Solders; spatiotemporal data; Spatiotemporal phenomena; Surface mount technology; surface mount technology (SMT)
• ISSN: 2168-2267; 2168-2275
• DOI: 10.1109/TCYB.2020.3033798

Surface mount technology (SMT) is a process for producing printed-circuit boards. The solder paste printer (SPP), package mounter, and solder reflow oven are used for SMT. The board on which the solder paste is deposited from the SPP is monitored by the solder paste inspector (SPI). If SPP malfunctions due to the printer defects, the SPP produces defective products, and then abnormal patterns are detected by SPI. In this article, we propose a convolutional recurrent reconstructive network (CRRN), which decomposes the anomaly patterns generated by the printer defects, from SPI data. CRRN learns only normal data and detects the anomaly pattern through the reconstruction error. CRRN consists of a spatial encoder (S-Encoder), a spatiotemporal encoder and decoder (ST-Encoder-Decoder), and a spatial decoder (S-Decoder). The ST-Encoder-Decoder consists of multiple convolutional spatiotemporal memories (CSTMs) with a spatiotemporal attention (ST-Attention) mechanism. CSTM is developed to extract spatiotemporal patterns efficiently. In addition, an ST-Attention mechanism is designed to facilitate transmitting information from the spatiotemporal encoder to the spatiotemporal decoder, which can solve the long-term dependency problem. We demonstrate that the proposed CRRN outperforms the other conventional models in anomaly detection. Moreover, we show the discriminative power of the anomaly map decomposed by the proposed CRRN through the printer defect classification.