Event based classification of Web 2.0 text streams

TL;DR

This paper proposes an event-based, real-time classification approach for Web 2.0 text streams like Twitter, using stream-dependent features and neural networks to enable immediate analysis and filtering of short-lived information.

Contribution

It introduces an event-based method for classifying Web 2.0 streams directly from data flow, enhancing real-time search capabilities over traditional storage-based approaches.

Findings

Event-based features effectively classify Twitter streams.

Neural networks support real-time filtering of short-lived data.

Stream-dependent features improve immediacy of information retrieval.

Abstract

Web 2.0 applications like Twitter or Facebook create a continuous stream of information. This demands new ways of analysis in order to offer insight into this stream right at the moment of the creation of the information, because lots of this data is only relevant within a short period of time. To address this problem real time search engines have recently received increased attention. They take into account the continuous flow of information differently than traditional web search by incorporating temporal and social features, that describe the context of the information during its creation. Standard approaches where data first get stored and then is processed from a peristent storage suffer from latency. We want to address the fluent and rapid nature of text stream by providing an event based approach that analyses directly the stream of information. In a first step we want to define the difference between real time search and traditional search to clarify the demands in modern text filtering. In a second step we want to show how event based features can be used to support the tasks of real time search engines. Using the example of Twitter we present in this paper a way how to combine an event based approach with text mining and information filtering concepts in order to classify incoming information based on stream features. We calculate stream dependant features and feed them into a neural network in order to classify the text streams. We show the separative capabilities of event based features as the foundation for a real time search engine.