ITran: A novel transformer-based approach for industrial anomaly detection and localization

• Published in: Engineering applications of artificial intelligence Vol. 125; p. 106677
• Main Authors: Cai, Xiangyu, Xiao, Ruliang, Zeng, Zhixia, Gong, Ping, Ni, Youcong
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.10.2023
• Subjects: Anomaly detection; Anomaly localization; Feature reconstruction; Industrial quality monitoring; Transformer; Anomaly localization; Transformer; Industrial quality monitoring; Feature reconstruction; Anomaly detection
• ISSN: 0952-1976; 1873-6769
• DOI: 10.1016/j.engappai.2023.106677

Anomaly detection is currently an essential quality monitoring process in industrial production. It is often affected by factors such as under or over reconstruction of images and unclear criteria for feature distribution evaluation, thus making it challenging to improve detection performance. To solve the above problems, this paper proposes a novel transformer-based approach, Inductive Transformer (ITran) for industrial anomaly detection and localization, which utilizes a multi-layer pyramid structure and multi-level jump connections to extract multi-scale features of the data, putting the anomaly detection into the feature space and achieving more accurate industrial anomaly detection and localization results. It incorporates inductive bias and convolution operations into the Transformer which helps to break the myth of Transformer being “data hungry”. Compared with the common Transformers, ITran significantly reduces the computational cost and memory usage and makes it work well on small datasets. In addition, we basically eliminate the effect of positional embedding on the proposed Transformer model. Sufficient experiments have been conducted to validate global anomaly detection on three datasets MNIST, Fashion-MNST and Cifar-10, as well as local anomaly detection on the industrial datasets MVTec AD, Concrete Crack Image and BTAD. The proposed ITran achieves outstanding results on all the above datasets. •Anomaly detection is an essential quality control process in industrial businesses.•The proposed ITran can be applied to both global and local anomaly detection.•Anomaly detection and localization work is put in the feature space.•The proposed method can precisely localize the defective regions.•The proposed method has low computational cost and better performance.