Inductive rule learning is arguably among the most traditional paradigms in machine learning. Although we have seen considerable progress over the years in learning rule-based theories, all state-of-the-art learners still learn descriptions that directly relate the input features to the target concept. It could nevertheless be the case that more structured representations, which form deep theories by forming intermediate concepts, could be easier to learn, in very much the same way as deep

more »

... l networks are able to outperform shallow networks, even though the latter are also universal function approximators. In this paper, we investigate into networks with weights and activations limited to the values 0 and 1. For the lack of a powerful algorithm that optimizes deep rule sets, we empirically compare deep and shallow rule networks with a uniform general algorithm, which relies on greedy mini-batch based optimization. Our experiments on both artificial and realworld benchmark data indicate that deep rule networks may outperform shallow networks.