Principled framework and technique for rule extraction from multi-layer perceptrons

The ability to extract rules from neural networks would enhance their value as classification systems by increasing the user's understanding of the problem domain. Recent activity has been focussed on either replacing the neural network by a rulebase, or using a network-rulebase hybrid to refine existing rules. This paper presents a framework for a classifier which contains a novelty detector, a multi-layer perceptron (MLP), and an extracted rulebase. The rulebase operates on a subset of non-novel patterns and is guaranteed to give the same classification as the MLP from which it was extracted. The rules are optimized with respect to parsimony, no specialized training of the MLP or discretization of input space is required, and a probabilistic interpretation is maintained throughout. The Fisher IRIS data is presented as a simple test case to demonstrate the validity of the approach.