Demo: TOSense -- What Did You Just Agree to?

TL;DR

TOSense is a Chrome extension that enables users to ask natural language questions about Terms of Service and receive concise, real-time answers by combining automated content extraction and a lightweight language model pipeline.

Contribution

It introduces a novel system integrating automated ToS crawling, a lightweight language model pipeline, and a synthetic question-answering evaluation method to improve ToS understanding.

Findings

Achieved up to 44.5% accuracy in real-world tests

Effectively extracts ToS content from major platforms

Demonstrates real-time, interactive ToS question answering

Abstract

Online services often require users to agree to lengthy and obscure Terms of Service (ToS), leading to information asymmetry and legal risks. This paper proposes TOSense-a Chrome extension that allows users to ask questions about ToS in natural language and get concise answers in real time. The system combines (i) a crawler "tos-crawl" that automatically extracts ToS content, and (ii) a lightweight large language model pipeline: MiniLM for semantic retrieval and BART-encoder for answer relevance verification. To avoid expensive manual annotation, we present a novel Question Answering Evaluation Pipeline (QEP) that generates synthetic questions and verifies the correctness of answers using clustered topic matching. Experiments on five major platforms, Apple, Google, X (formerly Twitter), Microsoft, and Netflix, show the effectiveness of TOSense (with up to 44.5% accuracy) across varying number of topic clusters. During the demonstration, we will showcase TOSense in action. Attendees will be able to experience seamless extraction, interactive question answering, and instant indexing of new sites.