Comprehensive Integration of Hyperdimensional Computing with Deep Learning towards Neuro-Symbolic AI

• Published in: 2023 60th ACM/IEEE Design Automation Conference (DAC) pp. 1 - 6
• Main Authors: Lee, Hyunsei, Kim, Jiseung, Chen, Hanning, Zeira, Ariela, Srinivasa, Narayan, Imani, Mohsen, Kim, Yeseong
• Format: Conference Proceeding
• Language: English
• Published: IEEE 09.07.2023
• Subjects: Costs; Deep learning; Design automation; Energy efficiency; Feature extraction; HD Computing; Knowledge Distillation; Memory management; Neuro-symbolic AI; Training
• DOI: 10.1109/DAC56929.2023.10248004

HD computing is a symbolic representation system which performs various learning tasks in a highly-parallelizable and binary-centric way by drawing inspiration from concepts in human long-term memory. However, the current HD computing is ineffective in extracting high-level feature information for image data. In this paper, we present a neuro-symbolic approach called NSHD, which integrates CNNs and Hyperdimensional (HD) learning techniques to provide efficient learning with state-of-the-art quality. We devise the HD training procedure, which fully integrates knowledge from the deep learning model through a distillation process with optimized computation costs due to the integration. Our experimental results show that NSHD provides high energy efficiency as compared to CNN, e.g., up to 64% with comparable accuracy, and can outperform the learning quality when more computing resources are allowed. We also show the symbolic nature of the NSHD can make the learning humnan-interpretable by exploiting the property of HD computing.