Representation Extraction Using Hyperbolic Knowledge Distilled Framework - An Industrial Application on High Risk Environment

• Published in: 2024 16th International Conference on Computer and Automation Engineering (ICCAE) pp. 163 - 167
• Main Authors: Kumar, Vijeth, Murugesan, Malathi, Veneri, Giacomo
• Format: Conference Proceeding
• Language: English
• Published: IEEE 14.03.2024
• Subjects: Contrastive Learning and Knowledge Distillation; Data models; Data Scarcity; Hyperbolic networks; Image Processing; Image Understanding; Knowledge engineering; Oil insulation; Representation learning; Service robots; Training; Transformers; Vision Transformers (ViTs)
• ISSN: 2154-4360
• DOI: 10.1109/ICCAE59995.2024.10569552

We propose a computer vision architecture based on Hyperbolic networks, contrastive learning and knowledge distillation to detect unsafe behavior in energy production and oil & gas plants. Data scarcity poses a significant challenge to develop machine learning applications in industry. Indeed, the data may be incomplete, inconsistent, or biased, making it difficult to develop accurate and reliable models. Insufficient data during training phase has direct impact on the models' representation learning capabilities; with the aid of Vision Transformers (ViTs), we are able to solve data crunch situations by learning efficient representations of the existing data. We harnessed the power of ViTs, as it incorporates more global information, leading to quantitatively stronger intermediate feature representations. Further, we approached the task with contrastive learning and obtained pairs of samples which are similar, to tackle the limited data availability in our industrial use case. The proposed approach by applying hyperbolic embeddings helps in extracting complex relationships in the data. Furthermore, the size of the model makes it suitable for devices with low computational capabilities such as unmanned robots.