Browser-Oriented Data Extraction

Abstract

Traditionally, most researchers used the URL-oriented data extraction model for data extraction. In this model, the systems extract URLs from pages and then use the extracted URLs to access next pages. However, more and more pages currently use script functions to access next pages. Since it is hard to extract URLs from script programs, it is inappropriate to use this model for such pages. For solving this problem, this paper proposed a new data extraction model, named the browseroriented data extraction model. In this model, the system built on top of browsers accesses pages by simulating users’ operations on browsers, which can also trigger script functions. Besides, this paper defines a scripting language, named the BODED (Browser-Oriented Data Extraction Description) Language, which instructs the system to do data extraction.