Learning to Map Query Terms to Document Categories is Adaptive Information Retrieval

Abstract

Information retrieval systems (IRS) often emplo inverted files to map query terms to those documents that contain the query terms. An inverted file consists of a set of terms and serves as an index to specific documents. However, selecting the terms and then mapping the terms to relevant documents are major bottlenecks. Manually selecting and map ping the terms often suffer from the problems of high cost and incomplete inverted files, since almost all terms (except for the small amount of stop words such as 'an' in English) may be meaningful to individual users. Furthermore, a document containing a term does not necessarily be relevant to the term. In this paper, we argue that there should be an incremental extensible inverted file to map query terms to their suitable document categories in which relevant documents are more likely to be found for the query. We propose a machine learning technique to acquire this kind of inverted files. The technique works on hierarchically structured text databases and acquires the way of mapping unknown terms to their suitable document categories. Thus the IRS ma adapt its search strategy to both the text database and the individual users' queries. This kind of adaptive information retrieval may promote both the quality and the efficiency of IRS, since full-text searching is conducted in suitable and smaller search spaces. The technique is theoretically evaluated. Its performance is empirically investigated using a real-world text database on the World Wide Web.