# Advocacy Learning: Learning through Competition and Class-Conditional Representations

## TL;DR

Advocacy learning introduces a competitive framework with class-conditional attention maps, where class-specific advocates aim to convince a judge of their class, leading to modest accuracy improvements and insights into class-conditional representations.

## Contribution

The paper proposes a novel advocacy learning scheme with class-conditional attention advocates and a judge, exploring their impact on classification performance and interpretability.

## Key findings

- Modest accuracy improvements over baseline models.
- Class-conditional representations can enhance discriminative performance.
- Competitive training of subnetworks provides a promising research direction.

## Abstract

We introduce advocacy learning, a novel supervised training scheme for attention-based classification problems. Advocacy learning relies on a framework consisting of two connected networks: 1) $N$ Advocates (one for each class), each of which outputs an argument in the form of an attention map over the input, and 2) a Judge, which predicts the class label based on these arguments. Each Advocate produces a class-conditional representation with the goal of convincing the Judge that the input example belongs to their class, even when the input belongs to a different class. Applied to several different classification tasks, we show that advocacy learning can lead to small improvements in classification accuracy over an identical supervised baseline. Though a series of follow-up experiments, we analyze when and how such class-conditional representations improve discriminative performance. Though somewhat counter-intuitive, a framework in which subnetworks are trained to competitively provide evidence in support of their class shows promise, in many cases performing on par with standard learning approaches. This provides a foundation for further exploration into competition and class-conditional representations in supervised learning.