# Multi-Observation Elicitation

## TL;DR

This paper introduces and analyzes multi-observation loss functions that evaluate predictions based on multiple data points simultaneously, potentially reducing complexity in property elicitation and machine learning.

## Contribution

It is the first to study multi-observation loss functions, showing their ability to lower hypothesis dimensionality and offering geometric insights into their properties.

## Key findings

- Multi-observation loss functions can significantly reduce hypothesis dimensionality.
- These functions enable fewer reports in elicitation and smaller hypothesis spaces in learning.
- Tradeoffs exist between the number of observations and the complexity of the elicited property.

## Abstract

We study loss functions that measure the accuracy of a prediction based on multiple data points simultaneously. To our knowledge, such loss functions have not been studied before in the area of property elicitation or in machine learning more broadly. As compared to traditional loss functions that take only a single data point, these multi-observation loss functions can in some cases drastically reduce the dimensionality of the hypothesis required. In elicitation, this corresponds to requiring many fewer reports; in empirical risk minimization, it corresponds to algorithms on a hypothesis space of much smaller dimension. We explore some examples of the tradeoff between dimensionality and number of observations, give some geometric characterizations and intuition for relating loss functions and the properties that they elicit, and discuss some implications for both elicitation and machine-learning contexts.