# Optimizing Majority Voting Based Systems Under a Resource Constraint for Multiclass Problems

## TL;DR

This paper explores how to select the most accurate ensemble of classifiers under resource constraints using a novel stochastic approach, addressing a complex optimization problem in multiclass classification.

## Contribution

It introduces a new stochastic method for optimizing ensemble accuracy under resource limits, with theoretical insights into multiclass ensemble performance.

## Key findings

- The stochastic approach effectively finds high-accuracy ensembles.
- Theoretical analysis provides bounds on ensemble accuracy and variance.
- Method applicable to binary and multiclass classification problems.

## Abstract

Ensemble-based approaches are very effective in various fields in raising the accuracy of its individual members, when some voting rule is applied for aggregating the individual decisions. In this paper, we investigate how to find and characterize the ensembles having the highest accuracy if the total cost of the ensemble members is bounded. This question leads to Knapsack problem with non-linear and non-separable objective function in binary and multiclass classification if the majority voting is chosen for the aggregation. As the conventional solving methods cannot be applied for this task, a novel stochastic approach was introduced in the binary case where the energy function is discussed as the joint probability function of the member accuracy. We show some theoretical results with respect to the expected ensemble accuracy and its variance in the multiclass classification problem which can help us to solve the Knapsack problem.