Using Full-Text Content to Characterize and Identify Best Seller Books

TL;DR

This study investigates whether the full text of books can predict their success as bestsellers, using visualization and classification methods on historical literary data, revealing limited predictability but offering insights into success factors.

Contribution

It is the first to analyze full-text content for bestseller prediction, combining visualization and machine learning to explore textual features related to success.

Findings

Logistic regression with bag-of-words achieved 75% accuracy.

Full text alone is insufficient for high-accuracy bestseller prediction.

Insights into factors influencing literary success were obtained.

Abstract

Artistic pieces can be studied from several perspectives, one example being their reception among readers over time. In the present work, we approach this interesting topic from the standpoint of literary works, particularly assessing the task of predicting whether a book will become a best seller. Dissimilarly from previous approaches, we focused on the full content of books and considered visualization and classification tasks. We employed visualization for the preliminary exploration of the data structure and properties, involving SemAxis and linear discriminant analyses. Then, to obtain quantitative and more objective results, we employed various classifiers. Such approaches were used along with a dataset containing (i) books published from 1895 to 1924 and consecrated as best sellers by the Publishers Weekly Bestseller Lists and (ii) literary works published in the same period but not being mentioned in that list. Our comparison of methods revealed that the best-achieved result - combining a bag-of-words representation with a logistic regression classifier - led to an average accuracy of 0.75 both for the leave-one-out and 10-fold cross-validations. Such an outcome suggests that it is unfeasible to predict the success of books with high accuracy using only the full content of the texts. Nevertheless, our findings provide insights into the factors leading to the relative success of a literary work.