Tiny-Engram: Trigger-Indexed Concept Tables for Generative Vision

TL;DR

Tiny-Engram introduces a compact, trigger-indexed concept table for generative vision models, enabling explicit lexical control over visual memories with promising results in image generation and potential in video personalization.

Contribution

The paper presents Tiny-Engram, a novel memory module that explicitly addresses concepts via trigger phrases, improving modularity and control in generative vision models.

Findings

Effective binding of trigger phrases to target identities in images.

Preserves compositional control from prompts in diffusion models.

Limited identity persistence in video generation, indicating room for improvement.

Abstract

Current personalization methods for generative vision models typically encode new concepts through continuous adapters or weight updates, yet provide limited control over whether and when a concept should be retrieved. In this work, we introduce Tiny-Engram, a compact trigger-indexed concept table that gives visual memories an explicit lexical address and activation boundary inside frozen image and video generators. Tiny-Engram parameterizes each concept as a small set of memory entries indexed by registered n-gram matches, which modulate text-encoder hidden states only within the matched trigger region. Outside this lexical support, the conditioning pathway is identical to that of the frozen base model. Across both single-encoder latent diffusion and multi-encoder diffusion-transformer backbones, this formulation binds a rare trigger phrase to a target identity while preserving compositional control from the surrounding prompt. We further evaluate the same table-based memory in a text-conditioned video generation setting, where the trigger path reliably alters the generated subject but fine-grained identity persistence across held-out video prompts remains limited. Taken together, these results suggest that small, explicitly addressed concept tables are a practical route to modular visual personalization, with strongest evidence in image generation. For video diffusion, the remaining gap points to a broader requirement: temporally stable identity likely depends on tighter coupling between text-side memory and the evolving visual state, motivating future work on memory injection beyond the text-conditioning interface.