# Adaptative Inference Cost With Convolutional Neural Mixture Models

## TL;DR

This paper introduces Convolutional Neural Mixture Models (CNMMs), a probabilistic framework that efficiently combines multiple CNNs to adapt inference costs dynamically, achieving high accuracy with flexible computational trade-offs.

## Contribution

The paper proposes CNMMs, a novel probabilistic model that enables dynamic pruning of CNN subsets for efficient inference without re-training.

## Key findings

- Achieves excellent accuracy-compute trade-offs in image classification and segmentation.
- Provides a wide range of operating points along the accuracy-cost spectrum.
- Allows inference cost adaptation without re-training.

## Abstract

Despite the outstanding performance of convolutional neural networks (CNNs) for many vision tasks, the required computational cost during inference is problematic when resources are limited. In this context, we propose Convolutional Neural Mixture Models (CNMMs), a probabilistic model embedding a large number of CNNs that can be jointly trained and evaluated in an efficient manner. Within the proposed framework, we present different mechanisms to prune subsets of CNNs from the mixture, allowing to easily adapt the computational cost required for inference. Image classification and semantic segmentation experiments show that our method achieve excellent accuracy-compute trade-offs. Moreover, unlike most of previous approaches, a single CNMM provides a large range of operating points along this trade-off, without any re-training.