# Using full-text content to characterize and identify best seller books: A study of early 20th-century literature

## TL;DR

This study explores how full-text content can help predict if a book will become a best seller, using data from early 20th-century literature.

## Contribution

The study uses full-text analysis and machine learning to predict best sellers, achieving 75% accuracy.

## Key findings

- Using full-text content and logistic regression achieved 75% accuracy in predicting best sellers.
- Visualization techniques like SemAxis helped explore data structure and properties.
- Predicting book success remains challenging despite using complete text data.

## Abstract

Artistic pieces can be studied from several perspectives, one example being their reception among readers over time. In the present work, we approach this interesting topic from the standpoint of literary works, particularly assessing the task of predicting whether a book will become a best seller. Unlike previous approaches, we focused on the full content of books and considered visualization and classification tasks. We employed visualization for the preliminary exploration of the data structure and properties, involving SemAxis and linear discriminant analyses. To obtain quantitative and more objective results, we employed various classifiers. Such approaches were used along with a dataset containing (i) books published from 1895 to 1923 and consecrated as best sellers by the Publishers Weekly Bestseller Lists and (ii) literary works published in the same period but not being mentioned in that list. Our comparison of methods revealed that the best-achieved result—combining a bag-of-words representation with a logistic regression classifier—led to an average accuracy of 0.75 both for the leave-one-out and 10-fold cross-validations. Such an outcome enhances the difficulty in predicting the success of books with high accuracy, even using the full content of the texts. Nevertheless, our findings provide insights into the factors leading to the relative success of a literary work.