Discovering Numerical-Type Dependencies for Improving the Accuracy of Decision Trees

Abstract

As we know, the decision tree learning algorithms, e.g., C5, are good at dataset classification. But those algorithms usually work with only one attribute at a time. The dependencies among attributes are not considered in those algorithms. Unfortunately, in the real world, most databases contain attributes, which are dependent. Thus, it is very important to construct a model to discovery the dependencies among attributes, and to improve the accuracy and effectiveness of the decision tree learning algorithms. Neural network model is a good choice for us to concern with the problems of attribute dependencies. Generally, these dependencies are classified into two types: categorical-type dependency and numerical-type dependency. This paper focuses on the numerical-type dependency and proposes a Neural Decision Tree (NDT) model, to deal with such kind of dependencies. The NDT model combines the neural network technologies and the traditional decision-tree learning capabilities, to handle the complicated and real cases. According to the experiments on five datasets from the UCI database repository, the NDT model can significantly improve the accuracy and effectiveness of C5.