Through the Twitter Glass: Detecting Questions in Micro-Text

TL;DR

This paper explores the challenge of detecting questions in Twitter micro-text using NLP techniques, highlighting the difficulties and potential strategies for processing Twitter data.

Contribution

It presents a pipeline of NLP tools tailored for Twitter to identify questions, demonstrating initial feasibility and lessons learned.

Findings

Twitter's idiosyncrasies complicate NLP processing

Simple syntactic structures in tweets may aid question detection

Preliminary results show potential for NLP in Twitter question detection

Abstract

In a separate study, we were interested in understanding people's Q&A habits on Twitter. Finding questions within Twitter turned out to be a difficult challenge, so we considered applying some traditional NLP approaches to the problem. On the one hand, Twitter is full of idiosyncrasies, which make processing it difficult. On the other, it is very restricted in length and tends to employ simple syntactic constructions, which could help the performance of NLP processing. In order to find out the viability of NLP and Twitter, we built a pipeline of tools to work specifically with Twitter input for the task of finding questions in tweets. This work is still preliminary, but in this paper we discuss the techniques we used and the lessons we learned.