Expert Graphs: Synthesizing New Expertise via Collaboration

TL;DR

This paper introduces 'expert graphs', a framework for analyzing and synthesizing expert opinions in classification tasks, establishing conditions for opinion consistency and predicting missing expertise.

Contribution

It formalizes expert opinion analysis using graph theory, deriving conditions for opinion consistency and creating synthetic experts, linking to the linear ordering polytope.

Findings

Derived necessary conditions for expert graph validity.

Created synthetic experts consistent with observed opinions.

Linked expert graphs to the linear ordering polytope.

Abstract

Consider multiple experts with overlapping expertise working on a classification problem under uncertain input. What constitutes a consistent set of opinions? How can we predict the opinions of experts on missing sub-domains? In this paper, we define a framework of to analyze this problem, termed "expert graphs." In an expert graph, vertices represent classes and edges represent binary opinions on the topics of their vertices. We derive necessary conditions for expert graph validity and use them to create "synthetic experts" which describe opinions consistent with the observed opinions of other experts. We show this framework to be equivalent to the well-studied linear ordering polytope. We show our conditions are not sufficient for describing all expert graphs on cliques, but are sufficient for cycles.