Foundational Study on Authorship Attribution of Japanese Web Reviews for Actor Analysis

TL;DR

This paper explores authorship attribution of Japanese web reviews using stylistic features, comparing methods like BERT fine-tuning and TF-IDF, with implications for actor analysis in threat intelligence.

Contribution

It evaluates various stylistic feature-based methods for Japanese review authorship attribution, highlighting their strengths and limitations for threat intelligence applications.

Findings

BERT fine-tuning achieved the highest accuracy but was unstable with many authors.

TF-IDF with logistic regression was more stable and cost-effective for larger author sets.

Error analysis identified boilerplate, topic dependency, and short texts as main misclassification factors.

Abstract

This study investigates the applicability of authorship attribution based on stylistic features to support actor analysis in threat intelligence. As a foundational step toward future application to dark web forums, we conducted experiments using Japanese review data from clear web sources. We constructed datasets from Rakuten Ichiba reviews and compared four methods: TF-IDF with logistic regression (TF-IDF+LR), BERT embeddings with logistic regression (BERT-Emb+LR), BERT fine-tuning (BERT-FT), and metric learning with $k$-nearest neighbors (Metric+kNN). Results showed that BERT-FT achieved the best performance; however, training became unstable as the number of authors scaled to several hundred, where TF-IDF+LR proved superior in terms of accuracy, stability, and computational cost. Furthermore, Top-$k$ evaluation demonstrated the utility of candidate screening, and error analysis revealed that boilerplate text, topic dependency, and short text length were primary factors causing misclassification.