Enhanced Out-of-Stock Detection in Retail Shelf Images Based on Deep Learning

• Published in: Sensors (Basel, Switzerland) Vol. 24; no. 2; p. 693
• Main Authors: Šikić, Franko, Kalafatić, Zoran, Subašić, Marko, Lončarić, Sven
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 01.01.2024; MDPI
• Subjects: Brand loyalty; Data collection; Datasets; Deep learning; Displays (Marketing); image analysis; image processing; Methods; out-of-stock detection; Radio frequency identification; Retail industry; Sensors; Stocks; Supermarkets; Europe; image processing; deep learning; out-of-stock detection; image analysis
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s24020693

The term out-of-stock (OOS) describes a problem that occurs when shoppers come to a store and the product they are seeking is not present on its designated shelf. Missing products generate huge sales losses and may lead to a declining reputation or the loss of loyal customers. In this paper, we propose a novel deep-learning (DL)-based OOS-detection method that utilizes a two-stage training process and a post-processing technique designed for the removal of inaccurate detections. To develop the method, we utilized an OOS detection dataset that contains a commonly used fully empty OOS class and a novel class that represents the frontal OOS. We present a new image augmentation procedure in which some existing OOS instances are enlarged by duplicating and mirroring themselves over nearby products. An object-detection model is first pre-trained using only augmented shelf images and, then, fine-tuned on the original data. During the inference, the detected OOS instances are post-processed based on their aspect ratio. In particular, the detected instances are discarded if their aspect ratio is higher than the maximum or lower than the minimum instance aspect ratio found in the dataset. The experimental results showed that the proposed method outperforms the existing DL-based OOS-detection methods and detects fully empty and frontal OOS instances with 86.3% and 83.7% of the average precision, respectively.