Solidago: A Modular Collaborative Scoring Pipeline

TL;DR

Solidago is a modular, secure, and scalable pipeline enabling communities to collaboratively score entities through trust-based user weighting, preference modeling, and secure aggregation, demonstrated on tournesol.app.

Contribution

It introduces a six-module decomposition for collaborative scoring, including novel trust propagation and integration of existing scaling and aggregation methods.

Findings

Successfully deployed on tournesol.app platform.

Provides a foundation for fair and interpretable collaborative scoring.

Includes novel trust propagation algorithm.

Abstract

This paper presents Solidago, an end-to-end modular pipeline to allow any community of users to collaboratively score any number of entities. Solidago proposes a six-module decomposition. First, it uses pretrust and peer-to-peer vouches to assign trust scores to users. Second, based on participation, trust scores are turned into voting rights per user per entity. Third, for each user, a preference model is learned from the user's evaluation data. Fourth, users' models are put on a similar scale. Fifth, these models are securely aggregated. Sixth, models are post-processed to yield human-readable global scores. We also propose default implementations of the six modules, including a novel trust propagation algorithm, and adaptations of state-of-the-art scaling and aggregation solutions. Our pipeline has been successfully deployed on the open-source platform tournesol.app. We thereby lay an appealing foundation for the collaborative, effective, scalable, fair, interpretable and secure scoring of any set of entities.