Predicting Oscar-Nominated Screenplays with Sentence Embeddings

TL;DR

This paper investigates predicting Oscar nominations for screenplays using sentence embeddings from modern language models, creating a new dataset and achieving promising classification results.

Contribution

It introduces the Movie-O-Label dataset and demonstrates that simple models with sentence embeddings can effectively predict Oscar nominations.

Findings

Best model achieved a macro F1 score of 0.66

Embedding combined screenplay, summary, and title improves performance

Model outperforms baseline with ROC-AUC of 0.79

Abstract

Oscar nominations are an important factor in the movie industry because they can boost both the visibility and the commercial success. This work explores whether it is possible to predict Oscar nominations for screenplays using modern language models. Since no suitable dataset was available, a new one called Movie-O-Label was created by combining the MovieSum collection of movie scripts with curated Oscar records. Each screenplay was represented by its title, Wikipedia summary, and full script. Long scripts were split into overlapping text chunks and encoded with the E5 sentence em bedding model. Then, the screenplay embed dings were classified using a logistic regression model. The best results were achieved when three feature inputs related to screenplays (script, summary, and title) were combined. The best-performing model reached a macro F1 score of 0.66, a precision recall AP of 0.445 with baseline 0.19 and a ROC-AUC of 0.79. The results suggest that even simple models based on modern text embeddings demonstrate good prediction performance and might be a starting point for future research.