Department of Computer Science
Discretizing numerical attributes in decision tree for big data analysis
© 2014 IEEE. The decision tree induction learning is a typical machine learning approach which has been extensively applied for data mining and knowledge discovery. For numerical data and mixed data, discretization is an essential pre-processing step of decision tree learning. However, when coping with big data, most of the existing discretization approaches will not be quite efficient from the practical viewpoint. Accordingly, we propose a new discretization method based on windowing and hierarchical clustering to improve the performance of conventional decision tree for big data analysis. The proposed method not only provides a faster process of discretizing numerical attributes with the competent classification accuracy, but also reduces the size of the decision tree. Experiments show the efficacy of the proposed method on the real data sets.
Big Data, Discretization, Hierarchical Clustering, Noise, Numerical Attribute, Window
Source Publication Title
2014 IEEE International Conference on Data Mining Workshop (ICDMW)
Zhang, Yiqun, and Yiu-Ming Cheung. "Discretizing numerical attributes in decision tree for big data analysis." 2014 IEEE International Conference on Data Mining Workshop (ICDMW) (2014): 1150-1157.