Unfair TOS: An Automated Approach using Customized BERT

TL;DR

This paper introduces a novel automated method using customized BERT fine-tuning combined with SVC to detect unfair clauses in Terms of Service documents, achieving state-of-the-art performance and providing a valuable tool for user protection.

Contribution

The study presents a new approach with customized BERT fine-tuning and SVC for unfair clause detection, outperforming existing methods and offering detailed analysis of transformer models.

Findings

Proficient macro F1-score of 0.922 in unfair clause detection

Superior performance in classifying unfair clauses by tags

Provides open-source code and results for further research

Abstract

Terms of Service (ToS) form an integral part of any agreement as it defines the legal relationship between a service provider and an end-user. Not only do they establish and delineate reciprocal rights and responsibilities, but they also provide users with information on essential aspects of contracts that pertain to the use of digital spaces. These aspects include a wide range of topics, including limitation of liability, data protection, etc. Users tend to accept the ToS without going through it before using any application or service. Such ignorance puts them in a potentially weaker situation in case any action is required. Existing methodologies for the detection or classification of unfair clauses are however obsolete and show modest performance. In this research paper, we present SOTA(State of The Art) results on unfair clause detection from ToS documents based on unprecedented custom BERT Fine-tuning in conjunction with SVC(Support Vector Classifier). The study shows proficient performance with a macro F1-score of 0.922 at unfair clause detection, and superior performance is also shown in the classification of unfair clauses by each tag. Further, a comparative analysis is performed by answering research questions on the Transformer models utilized. In order to further research and experimentation the code and results are made available on https://github.com/batking24/Unfair-TOS-An-Automated-Approach-based-on-Fine-tuning-BERT-in-conjunction-with-ML.