MemCam: Memory-Augmented Camera Control for Consistent Video Generation

TL;DR

MemCam introduces a memory-augmented approach for interactive video generation that maintains scene consistency over long sequences by leveraging external memory and context compression.

Contribution

The paper proposes MemCam, a novel memory-augmented method with context compression and dynamic retrieval to improve scene consistency in long, camera-controlled videos.

Findings

MemCam outperforms baselines in scene consistency for long videos.

It effectively maintains scene coherence during large camera rotations.

The approach reduces computational overhead while enriching contextual information.

Abstract

Interactive video generation has significant potential for scene simulation and video creation. However, existing methods often struggle with maintaining scene consistency during long video generation under dynamic camera control due to limited contextual information. To address this challenge, we propose MemCam, a memory-augmented interactive video generation approach that treats previously generated frames as external memory and leverages them as contextual conditioning to achieve controllable camera viewpoints with high scene consistency. To enable longer and more relevant context, we design a context compression module that encodes memory frames into compact representations and employs co-visibility-based selection to dynamically retrieve the most relevant historical frames, thereby reducing computational overhead while enriching contextual information. Experiments on interactive video generation tasks show that MemCam significantly outperforms existing baseline methods as well as open-source state-of-the-art approaches in terms of scene consistency, particularly in long video scenarios with large camera rotations.