Robust Evaluation of Neural Encoding Models via ground-truth approximation

TL;DR

This paper introduces a new evaluation framework for neural encoding models that approximates ground-truth neural activity, significantly improving sensitivity and robustness over traditional metrics.

Contribution

It proposes a ground-truth approximation method using canonical correlation analysis and participant averaging, enhancing model evaluation accuracy.

Findings

CPA-PA metric outperforms conventional scores by 300-1000% on synthetic EEG data.

On 34 real MEEG datasets, CPA-PA improves evaluation sensitivity by 250%.

The framework reduces dependence on SNR and better captures stimulus-relevant neural activity.

Abstract

Encoding models enable measurement of how our brains represent sensory inputs using electro-and magneto-encephalography (MEEG). Evaluating how closely encoding models reflect the underlying brain functions is a crucial premise for model interpretation and hypothesis testing. However, the ground-truth neural activity is unknown, preventing model evaluation with respect to the target neural signal. Existing evaluation metrics must therefore relate model's predictions to noisy MEEG measurements, where most variance is stimulus-unrelated. Here, I introduce an evaluation framework where model predictions are compared to a ground-truth approximation, obtained by aligning MEEG signals with predictions using canonical correlation analysis and via participant averaging. The resulting metric (CPA-PA) yields single-participant evaluations outperforming conventional scores by 300-1000% on synthetic EEG data and 250% on 34 real MEEG datasets (818 datapoints). These gains reflect increased sensitivity to stimulus-relevant neural activity and reduced dependence on SNR, establishing ground-truth approximation as a robust framework for evaluating encoding models.