Four Eyes Are Better Than Two: Harnessing the Collaborative Potential of Large Models via Differentiated Thinking and Complementary Ensembles

TL;DR

This paper demonstrates how leveraging differentiated prompting and ensemble strategies with large multimodal models significantly improves video understanding performance, outperforming previous state-of-the-art methods.

Contribution

The paper introduces a novel approach combining prompt diversification and model ensembling to enhance large model performance on video understanding tasks.

Findings

Direct use of a single multimodal model surpasses previous SOTA.

Ensemble of periodic results further boosts performance.

Systematic exploration of prompt styles improves model guidance.

Abstract

In this paper, we present the runner-up solution for the Ego4D EgoSchema Challenge at CVPR 2025 (Confirmed on May 20, 2025). Inspired by the success of large models, we evaluate and leverage leading accessible multimodal large models and adapt them to video understanding tasks via few-shot learning and model ensemble strategies. Specifically, diversified prompt styles and process paradigms are systematically explored and evaluated to effectively guide the attention of large models, fully unleashing their powerful generalization and adaptability abilities. Experimental results demonstrate that, with our carefully designed approach, directly utilizing an individual multimodal model already outperforms the previous state-of-the-art (SOTA) method which includes several additional processes. Besides, an additional stage is further introduced that facilitates the cooperation and ensemble of periodic results, which achieves impressive performance improvements. We hope this work serves as a valuable reference for the practical application of large models and inspires future research in the field. Our Code is available at https://github.com/XiongjunGuan/EgoSchema-CVPR25.