PrisCrawler: A Relevance Based Crawler for Automated Data Classification from Bulletin Board

TL;DR

Priscrawler is an automated system that enhances bulletin board search engines by classifying attachments based on relevance, improving search efficiency and accuracy through an innovative relevance algorithm.

Contribution

The paper introduces Priscrawler, a novel subsystem utilizing Attachrank to automatically classify and relate attachments, simplifying search and increasing precision.

Findings

Effective classification of attachments demonstrated

Improved search precision confirmed by experiments

Reduced complexity of search subsystems

Abstract

Nowadays people realize that it is difficult to find information simply and quickly on the bulletin boards. In order to solve this problem, people propose the concept of bulletin board search engine. This paper describes the priscrawler system, a subsystem of the bulletin board search engine, which can automatically crawl and add the relevance to the classified attachments of the bulletin board. Priscrawler utilizes Attachrank algorithm to generate the relevance between webpages and attachments and then turns bulletin board into clear classified and associated databases, making the search for attachments greatly simplified. Moreover, it can effectively reduce the complexity of pretreatment subsystem and retrieval subsystem and improve the search precision. We provide experimental results to demonstrate the efficacy of the priscrawler.