# Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction

## TL;DR

This paper introduces a novel framework for multimodal future prediction that generates multiple future samples, groups them into modes, and evaluates the distributions effectively, overcoming limitations of existing mixture density approaches.

## Contribution

It proposes a sampling and grouping method with a winner-takes-all loss for stable multimodal future prediction, addressing mode collapse and training instability issues.

## Key findings

- Successfully predicts multimodal distributions on synthetic data
- Avoids mode collapse in real-world scenarios
- Provides effective evaluation methods for multimodal predictions

## Abstract

Future prediction is a fundamental principle of intelligence that helps plan actions and avoid possible dangers. As the future is uncertain to a large extent, modeling the uncertainty and multimodality of the future states is of great relevance. Existing approaches are rather limited in this regard and mostly yield a single hypothesis of the future or, at the best, strongly constrained mixture components that suffer from instabilities in training and mode collapse. In this work, we present an approach that involves the prediction of several samples of the future with a winner-takes-all loss and iterative grouping of samples to multiple modes. Moreover, we discuss how to evaluate predicted multimodal distributions, including the common real scenario, where only a single sample from the ground-truth distribution is available for evaluation. We show on synthetic and real data that the proposed approach triggers good estimates of multimodal distributions and avoids mode collapse. Source code is available at $\href{https://github.com/lmb-freiburg/Multimodal-Future-Prediction}{\text{this https URL.}}$