Automatic Clustering of Web News Based on Topics-Discovery

Abstract

This paper proposes a new method for the unsupervised clustering of large and high-dimensional sets of textual data. The system begins with the topics-discovery process, which determines the k groups of document with maximal intra-group similarity and well scattered throughout the similarity space of the text collection. These k document groups are regarded as the central topics of the entire document collection. Then an intelligent feature selection algorithm is applied to deriving the features, called as topic keywords, that are the most suitable representation of the topics. Finally, all documents in the collection are clustered into k clusters according to the topic keywords. This method provides advantages of a very efficient clustering operation and involves no humanly predefined thresholds, which mean that no expert intervention is required. The experimental results indicate that this approach generated higher quality of cluster than many well-known document clustering algorithms.