When Fusion Helps and When It Breaks: View-Aligned Robustness in Same-Source Financial Imaging

TL;DR

This paper investigates the robustness of multi-view financial image data fusion methods against adversarial attacks, revealing that late fusion generally offers more reliable robustness than early fusion under various threat models.

Contribution

It provides a comprehensive comparison of early and late fusion strategies in multi-view financial imaging, highlighting their robustness and vulnerabilities against adversarial attacks.

Findings

Late fusion is more reliable than early fusion under noisy conditions.

Robustness sharply decreases with small attack budgets.

View-dependent vulnerabilities persist under joint perturbations.

Abstract

We study same-source multi-view learning and adversarial robustness for next-day direction prediction using two deterministic, window-aligned image views derived from the same time series: an OHLCV-rendered chart (ohlcv) and a technical-indicator matrix (indic). To control label ambiguity from near-zero moves, we use an ex-post minimum-movement threshold min_move (tau) based on realized absolute next-day return, defining an offline benchmark on the subset where the absolute next-day return is at least tau. Under leakage-resistant time-block splits with embargo, we compare early fusion (channel stacking) and dual-encoder late fusion with optional cross-branch consistency. We then evaluate pixel-space L-infinity evasion attacks (FGSM/PGD) under view-constrained and joint threat models. We find that fusion is regime dependent: early fusion can suffer negative transfer under noisier settings, whereas late fusion is a more reliable default once labels stabilize. Robustness degrades sharply under tiny budgets with stable view-dependent vulnerabilities; late fusion often helps under view-constrained attacks, but joint perturbations remain challenging.