Adopting attention-mechanisms for Neural Logic Rule Layers

In previous works we discovered that rule-based systems severely suffer in performance when increasing the number of rules. In order to increase the amount of possible boolean relations while keeping the number of rules fixed, we employ ideas from well known Spatial Transformer Systems and Self-Atte...

Full description

Saved in:
Bibliographic Details
Published inAutomatisierungstechnik : AT Vol. 70; no. 3; pp. 257 - 266
Main Authors Reimann, Jan Niclas, Schwung, Andreas, Ding, Steven X.
Format Journal Article
LanguageEnglish
Published De Gruyter Oldenbourg 28.03.2022
Subjects
Gewichtsprediktion
interpretability
Interpretierbarkeit
Neural Logic Rule Layers
Neurale Regelbasierte Netze
weight prediction
Online AccessGet full text

Cover

More Information
Summary:In previous works we discovered that rule-based systems severely suffer in performance when increasing the number of rules. In order to increase the amount of possible boolean relations while keeping the number of rules fixed, we employ ideas from well known Spatial Transformer Systems and Self-Attention Networks: here, our learned rules are not static but are dynamically adjusted to fit the input data by training a separate rule-prediction system, which is predicting parameter matrices used in Neural Logic Rule Layers. We show, that these networks, termed Adaptive Neural Logic Rule Layers, outperform their static counterpart both in terms of final performance, as well as training stability and excitability during early stages of training.
ISSN:0178-2312
2196-677X
DOI:10.1515/auto-2021-0136