Camera-based Progress Estimation of Assembly Work Using Deep Metric Learning

In this paper, a progress estimation method using deep learning is proposed to visualize the product assembly process in a factory. First, the target assembly product is detected from images acquired from a fixed-point camera installed in the factory using a deep learning-based object detection meth...

Full description

Saved in:
Bibliographic Details
Published in2023 IEEE/SICE International Symposium on System Integration (SII) pp. 1 - 6
Main Authors Kitsukawa, Takumi, Pathak, Sarthak, Moro, Alessandro, Harada, Yoshihiro, Nishikawa, Hideo, Noguchi, Minori, Hamaya, Akifumi, Umeda, Kazunori
Format Conference Proceeding
LanguageEnglish
Published IEEE 17.01.2023
Subjects
Deep learning
Estimation
Focusing
Measurement
Object detection
System integration
Visualization
Online AccessGet full text

Cover

More Information
Summary:In this paper, a progress estimation method using deep learning is proposed to visualize the product assembly process in a factory. First, the target assembly product is detected from images acquired from a fixed-point camera installed in the factory using a deep learning-based object detection method. Next, the detection area is cropped from the image. Finally, by using a classification method based on deep metric learning on the cropped image, the progress of the product assembly work is estimated as a rough progress step. In addition, considering the similarity of features with neighboring steps when learning with deep metric learning, we propose an adaptive loss function that learns to separate features from nearby steps. In experiments, an 82 [%] success rate is achieved for the progress estimation method using deep metric learning. Furthermore, the method using the adaptive loss function achieved a success rate of 92 [%]. Experiments were also conducted to verify the practicality of a series of detection, cropping and progress estimation.
ISSN:2474-2325
DOI:10.1109/SII55687.2023.10039109