An Empirical Investigation Into Deep and Shallow Rule Learning
Inductive rule learning is arguably among the most traditional paradigms in machine learning. Although we have seen considerable progress over the years in learning rule-based theories, all state-of-the-art learners still learn descriptions that directly relate the input features to the target conce...
Saved in:
Published in | Frontiers in artificial intelligence Vol. 4; p. 689398 |
---|---|
Main Authors | , |
Format | Journal Article |
Language | English |
Published |
Switzerland
Frontiers Media S.A
22.10.2021
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Inductive rule learning is arguably among the most traditional paradigms in machine learning. Although we have seen considerable progress over the years in learning rule-based theories, all state-of-the-art learners still learn descriptions that directly relate the input features to the target concept. In the simplest case, concept learning, this is a disjunctive normal form (DNF) description of the positive class. While it is clear that this is sufficient from a logical point of view because every logical expression can be reduced to an equivalent DNF expression, it could nevertheless be the case that more structured representations, which form deep theories by forming intermediate concepts, could be easier to learn, in very much the same way as deep neural networks are able to outperform shallow networks, even though the latter are also universal function approximators. However, there are several non-trivial obstacles that need to be overcome before a sufficiently powerful deep rule learning algorithm could be developed and be compared to the state-of-the-art in inductive rule learning. In this paper, we therefore take a different approach: we empirically compare deep and shallow rule sets that have been optimized with a uniform general mini-batch based optimization algorithm. In our experiments on both artificial and real-world benchmark data, deep rule networks outperformed their shallow counterparts, which we take as an indication that it is worth-while to devote more efforts to learning deep rule structures from data. |
---|---|
Bibliography: | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 Edited by: Sophie Burkhardt, Johannes Gutenberg University Mainz, Germany Joao Gama, University of Porto, Portugal Reviewed by: Fabio Aiolli, University of Padua, Italy This article was submitted to Machine Learning and Artificial Intelligence, a section of the journal Frontiers in Artificial Intelligence |
ISSN: | 2624-8212 2624-8212 |
DOI: | 10.3389/frai.2021.689398 |