DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer

TL;DR

DOPRA is a decoding-based method that reduces hallucinations in multi-modal large language models by applying layer-specific penalties and reallocation strategies during decoding, without needing extra training data.

Contribution

It introduces a novel decoding technique that mitigates hallucinations in MLLMs through layer-specific penalties and token reallocation, avoiding additional training or external knowledge.

Findings

Reduces hallucinations in MLLMs during decoding.

Improves alignment of generated content with actual image data.

Does not require extra training data or external resources.

Abstract

In this work, we introduce DOPRA, a novel approach designed to mitigate hallucinations in multi-modal large language models (MLLMs). Unlike existing solutions that typically involve costly supplementary training data or the integration of external knowledge sources, DOPRA innovatively addresses hallucinations by decoding specific weighted layer penalties and redistribution, offering an economical and effective solution without additional resources. DOPRA is grounded in unique insights into the intrinsic mechanisms controlling hallucinations within MLLMs, especially the models' tendency to over-rely on a subset of summary tokens in the self-attention matrix, neglecting critical image-related information. This phenomenon is particularly pronounced in certain strata. To counteract this over-reliance, DOPRA employs a strategy of weighted overlay penalties and redistribution in specific layers, such as the 12th layer, during the decoding process. Furthermore, DOPRA includes a retrospective allocation process that re-examines the sequence of generated tokens, allowing the algorithm to reallocate token selection to better align with the actual image content, thereby reducing the incidence of hallucinatory descriptions in auto-generated captions. Overall, DOPRA represents a significant step forward in improving the output quality of MLLMs by systematically reducing hallucinations through targeted adjustments during the decoding process.