Rapid Prototyping of a Text Mining Application for Cryptocurrency Market Intelligence

TL;DR

This paper presents a rapid prototyping pipeline for analyzing social media data related to blockchain and cryptocurrency using NLP and ML, demonstrating preliminary results and scalability plans.

Contribution

It introduces a scalable proof-of-concept system for real-time social media analysis of cryptocurrency trends using advanced analytics.

Findings

Preliminary analysis shows promising performance metrics.

System architecture supports multi-stream data processing.

Future plans include cloud-based real-time analysis with IE and ML.

Abstract

Blockchain represents a technology for establishing a shared, immutable version of the truth between a network of participants that do not trust one another, and therefore has the potential to disrupt any financial or other industries that rely on third-parties to establish trust. Recent trends in computing including: prevalence of Free and Open Source Software (FOSS); easy access to High Performance Computing (HPC i.e. 'The Cloud'); and increasingly advanced analytics capabilities such as Natural Language Processing (NLP) and Machine Learning (ML) allow for rapidly prototyping applications for analysis of trends in the emergence of Blockchain technology. A scaleable proof-of-concept pipeline that lays the groundwork for analysis of multiple streams of semi-structured data posted on social media is demonstrated. Preliminary analysis and performance metrics are presented and discussed. Future work is described that will scale the system to cloud-based, real-time, analysis of multiple data streams, with Information Extraction (IE) (ex. sentiment analysis) and Machine Learning capability.