Investigating Classification Techniques with Feature Selection For Intention Mining From Twitter Feed

TL;DR

This paper explores feature selection methods for classifying user intentions from Twitter feeds, comparing a one-phase approach with a hybrid method using multiple classifiers on a custom dataset.

Contribution

It introduces a hybrid feature selection approach combining multiple techniques and evaluates their effectiveness for intention classification from Twitter data.

Findings

Hybrid feature selection improves classification accuracy.

Information Gain-based selection is effective for text data.

Different classifiers show varying performance with selected features.

Abstract

In the last decade, social networks became most popular medium for communication and interaction. As an example, micro-blogging service Twitter has more than 200 million registered users who exchange more than 65 million posts per day. Users express their thoughts, ideas, and even their intentions through these tweets. Most of the tweets are written informally and often in slang language, that contains misspelt and abbreviated words. This paper investigates the problem of selecting features that affect extracting user's intention from Twitter feeds based on text mining techniques. It starts by presenting the method we used to construct our own dataset from extracted Twitter feeds. Following that, we present two techniques of feature selection followed by classification. In the first technique, we use Information Gain as a one-phase feature selection, followed by supervised classification algorithms. In the second technique, we use a hybrid approach based on forward feature selection algorithm in which two feature selection techniques employed followed by classification algorithms. We examine these two techniques with four classification algorithms. We evaluate them using our own dataset, and we critically review the results.