Social learning of emotion and its implication for memory: an ERP study
Sriranjani Manivasagam, Anne Schacht
TL;DR
This study explores how social cues influence emotional learning and memory using EEG, finding that their impact depends on how certain people feel about what they see.
Contribution
The study reveals how social cues modulate emotional salience and memory encoding based on subjective perceptual uncertainty.
Findings
P1 amplitudes in the learning session reflected modulation by subjective perceptual uncertainty.
EPN amplitudes were enhanced for positive images paired with social cues on the test day.
LPC amplitudes were reduced for negative images in the social cue condition, regardless of accuracy.
Abstract
Social learning of emotional salience from surrounding social cues is particularly advantageous under conditions of uncertainty. Yet, the neural mechanisms underlying this process and its consolidation into long-term memory remain poorly understood. In this two-day EEG study, we examined whether emotional salience from social cues (facial expressions) transfers to perceptually uncertain target images, and whether such learned salience is preserved in memory even after the social cues are removed. On Day 1 (learning session), preregistered analyses provided no evidence for an automatic emotional salience transfer across trials under the task’s uniform perceptual uncertainty. Instead, exploratory ERP analyses indicated that the use of social cues depended on subjective perceptual uncertainty, indexed by participants’ classification accuracy of the target image. P1 amplitudes in the…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14- —Georg-August-Universität Göttingen (1018)
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace Recognition and Perception · Neural and Behavioral Psychology Studies · Memory Processes and Influences
Introduction
Humans are inherently social beings, thereby making social cues such as facial or vocal expressions, gaze, gestures, postures etc. prevalent in the human environment. Social cues enable social learning through the observation and imitation of others, without the need for direct first-hand experiences^1–4^. Social learning is assumed to play an indispensable role from early ontogenetic development to the emergence of shared knowledge, ultimately leading to cultural evolution^5–7^. The foundations of social learning can be traced back to the origins of social cognition, leading to primates’ sophisticated abilities to recognize social cues from conspecifics^8,9^. Social cues contain a rich variety of information such as a social agent’s object of attention, their intentions and goals^10^. Recognizing, interpreting and learning from social cues support adaptive behavior for reward acquisition and survival^11–14^. Social cues trigger enhanced neural processing compared to non-social cues, particularly in brain areas involved in early attention and performance monitoring^15,16^. Orientation to visual social cues also recruits more cortical structures of the brain, such as the extrastriate, which is key to adapting future responses to environmental objects^17,18^. Studies on social attention have further pointed out overt shifts of attention towards social cues^10,19–22^. Taken together, the findings indicate that social cues hold a high motivational value, rendering them salient. Particularly, a class of social cues such as facial expressions or vocal sounds (also called affect bursts), conveys rich emotion-related information not only about the social agent’s mental state but also about the value associated with environmental objects (target stimuli). In this study, we examined emotional salience, which arises from social cues such as facial expressions. We further investigated the neural dynamics of social learning of emotional salience using images that depict various environmental objects and scenes, such as flowers, car-crashes, household objects, etc. as target stimuli, and its preservation in memory.
Traditionally, social learning has been suggested to fall within three categories: stimulus enhancement, observational conditioning, and observational learning (imitation and emulation). Consider an example of a person (social agent) reading a book (stimulus) in the garden and smiling (emotionally salient social cue). Stimulus enhancement takes place when an observer’s attention is directed towards the book following the presence of a smile, making the book salient^23,24^. Observational conditioning occurs when an observer witnesses the co-occurrence of the book and the smile, subsequently getting conditioned to expect a positive outcome from the book^25–27^. The third category of social learning is observational learning. It involves a change in behavior by imitating the action of the social agent, which in the example above would be to read the same book in the garden. Observational learning also involves emulation, which focuses on the outcome without necessarily copying the actions of the social agent – in this case one could obtain and read the book by visiting a library^28,29^. A family of mechanisms that subsumes the above categories of social learning has been proposed as associative learning processes^30–32^. Thus, using associative processes, social learning has been suggested to follow fundamental mechanisms similar to those of non-social learning, with the key difference being the input that drives the learning process, i.e. social cues^31^. Assuming that associative mechanisms are the underlying principle of social learning, we hypothesized that emotional salience is transferred from social cues to target stimuli, enabling adaptive responses towards the target stimuli.
Previous research has focused on the social appraisal process, in which different social cues influence the evaluation of emotional salience of the target stimuli^20,21,33,34^. Using the social appraisal conceptual framework, the neural modulations involved in appraising target stimuli based on the emotional salience of social cues have been studied. The appraisal-focused studies allude to expectation-based modulations at the neural level or an instructed perspective-taking strategy, where the participants view the target stimuli from the perspective of the other person (social agent). Although appraisal strategies lead to adaptive responses to the target stimuli, they have been only investigated on a single trial basis. The dynamics and development of the learning of emotional salience over time have been largely neglected^35–38^. Moreover, the preservation of associated emotional salience in target stimuli in the subsequent absence of social cues have not been investigated. An associative learning framework allows for the investigation of the evolution of emotional salience transfer and its potential long-term maintenance while dissociating the direct effects of the social cues themselves.
Several previous studies have focused on the transfer of emotional salience, often through monetary gains or losses, to neutral stimuli – such as images of scenes, faces, linguistic materials, and abstract symbols that either do not carry inherent emotional salience or are normatively rated to be less emotionally salient. Event-related potential (ERP) evidence indicates that the acquisition of emotional salience associations result in prioritized processing of previously neutral or meaningless stimuli at both early perceptual and subsequent higher-order stages through an assumed ‘value-based attention mechanism’^39^. Further, the effect of transferred emotional salience has been observed during test phases, either on the same day or one day after the learning of associations in behavioral measures as well as in short-^40–43^, mid- and long-latency ERPs^40,44^. Notably, similar ERP modulations of the transferred emotional salience have been reported for testing with a one-week delay, indicating a persistence of associated emotional salience in long-term memory^45^.
However, the influence of social cues extends beyond neutral target stimuli to instances, where the target stimuli inherently carry emotional salience, for example due to biological relevance or previous experiences. Therefore, to investigate the transfer of emotional salience from social cues to target stimuli, it is essential to consider the presence of emotional salience from both social cues and the target stimuli. In this study, we examined emotionally salient (happy, sad) and neutral facial expressions as social cues and images that are emotionally salient (positive, negative) and neutral as target stimuli. Neuroscientific research has indicated an identical potency of both emotionally salient facial expressions and images to elicit typical emotion-related ERP markers^46^. For example, the Early Posterior Negativity (EPN), an ERP component observed at temporal-occipital sites approximately 250 ms after the stimulus onset, has been reported to be augmented in response to both emotionally salient facial expressions and images, presumably reflecting increased selective attention^46–51^. The EPN is typically followed by the Late Positive Complex (LPC), also named Late Positive Potential, over centro-parietal electrodes, starting around 300 ms after stimulus onset^49,52–55^. This long-lasting ERP response has been assumed to reflect higher-order elaborate and evaluative processes^46,48^. In addition, some studies have also reported enhanced P1 amplitudes to both emotionally salient facial expressions and images, peaking around 100 ms after stimulus onset and consisting of bilateral occipital positivities. P1 modulations have been functionally linked to the activation of extrastriate visual areas through selective attention at early perceptual stages of stimulus processing^48,56–58^. In this study, we employed ERPs to investigate the neural dynamics of the interaction between the inherent emotional salience of the images and the emotional salience transferred from the facial expressions.
A successful transfer of emotional salience from the social cues to the target stimuli requires two fundamental prerequisites: first, the motivation of the observer to learn from social cues, and second, a certain degree of uncertainty in predicting the emotional salience outcome of the target stimuli. For example, novelty, ambiguity, or perceptual uncertainty (due to less reliable perceptual information) results in a lack of clarity when making judgments about the emotional salience of the target stimuli. Along similar lines, previous studies suggested that social cues are often attended to gain more information about the environment, and the employment of such cues depends on several contextual factors, including prevailing uncertainty, the desire to affiliate with a particular social group, or the judged competitiveness of the social agent providing the cue^59–61^. Particularly in such contexts, the emotional salience of the target stimuli can be determined by integrating emotionally salient social cues from the environment^62–64^, leading to behavioral and neural facilitation^65,66^. Consequently, social learning has been proposed to be a meta-cognitive strategy in humans and other animals^4,60,67^. Studies have demonstrated that social cues strongly impact early perceptual processing, especially in the presence of the contextual factors mentioned above. Additionally, research on multisensory integration provides evidence that contextual cues are integrated, resulting in prioritized processing at early stages when the target stimuli are presented under perceptual-uncertainty^66^. Supporting the aforementioned evidence, a study found that social cues can drive selective attention to perceptually uncertain stimuli already at the initial stages of processing^68^. In addition, previous studies have identified activations in the lower-level processing areas to enable group conformity^69,70^. A similar observation has been made in an ERP-based study under group pressure, where social cues can strongly impact participants, as evidenced by increased P1 amplitudes in early visual processing when participants aligned with the correct group opinion^71^. Interestingly, the same study also reported a reduction in P1 amplitudes when participants conformed to an incorrect group opinion, suggesting reduced stimulus processing and discrimination. Based on the above evidence, we expected external motivations in the form of perceptual uncertainty or conformity pressure to result in modulations at the early perceptual stages and higher-order stimulus evaluations.
The present study builds on prior evidence that uncertainty fosters a propensity for social learning. It examines how emotionally salient social cues (facial expressions) influence judgments about the emotional salience of target stimuli (target images). In a learning session, participants performed an explicit judgment of emotional salience of the target images, that is, the participants classified the target images as positive, negative or neutral in a valence classification task under experimentally induced perceptual uncertainty. Perceptual uncertainty was induced by a forward mask that was followed by a brief stimulus presentation for 27 ms. The visual processing of the forward mask competed with the target image and resulted in perceptual uncertainty when judging the valence of the target image (see Supplementary Information 1).
We further investigated whether the learned associations, based on valence-congruent combinations of target images and facial expressions during learning, persisted in memory following overnight consolidation, even when the facial expressions were no longer provided in the test session.
The learning session consisted of an explicit valence classification task of briefly presented target images with positive, negative, or neutral valence. The target image was followed by (a) happy, sad, or neutral facial expression that was introduced as the emotional responses of previous participants to a certain image (social cue condition), or (b) a scrambled face (control cue condition). Due to the perceptual uncertainty in the valence classification task, we expected the facial expressions to provide relevant information, resulting in the learning of the target image-facial expression associations. In the test session, an Old/New classification task required decisions on the previously learned target images (old) and distractor images (new), which varied in their emotional salience. The comparison of (a) the old target images associated with the facial expressions and (b) the old target images associated with the scrambled faces allowed estimating the transfer and preservation of emotional salience in the absence of facial expressions. By monitoring task performance and computing ERPs, we aimed to investigate the behavioral and neural modulation of the hypothesized emotional salience transfer from facial expressions towards the target images and its consequences for subsequent memory.
Hypotheses (see Table 1. design table)
Table 1. Design Table with detailed hypothesis, sampling and analysis plan with interpretations for each research question.QuestionHypothesis (if applicable)Sampling plan (e.g., power analysis)Analysis PlanInterpretation given to different outcomes Research Question 1. Does the emotional salience of the facial expressions integrate with that of the target images under perceptual uncertainty during the learning session, and if so, what are the neuro-cognitive mechanisms underlying this emotional salience transfer?Null Hypothesis (H0) for Behavioral Performance in the Valence Classification Task and ERP amplitudes:No difference between cue conditions (social/control) as a result of learning. Only effects of target image valence – i.e., emotional compared to neutral images.H0: Accuracy/ERP amplitudes: (Emotional images_Social = = Emotional images_Control) > (Neutral images_Social = = Neutral images_Control)H0: RTs: (Emotional images_Social = = Emotional images_Control) < (Neutral images_Social = = Neutral images_Control) Hypothesis 1: Valence Classification Accuracy Earlier trials (Comparable effects across valence in the beginning),Accuracy: (Emotional images_Social = = Neutral images_Social) > (Emotional images_Control = = Neutral images_Control)Later trials,Accuracy: Emotional images_Social > Emotional images_Control > Neutral images_Social > Neutral images_ControlHypothesis 2: Valence Classification RTs:Earlier trials,RTs: (Emotional images_Social = = Neutral images_Social) < (Emotional images_Control = = Neutral images_Control)Later trials,RTs: (Emotional images_Social < Emotional images_Control) < (Neutral images_Social < Neutral images_Control) Hypothesis 3: ERP amplitudes (time locked to images) Earlier trials (Comparable effects across cue conditions in the beginning),ERP amplitudes (P1/EPN/LPC): (Emotional images_Social = = Emotional images_ Control) > (Neutral images_Social = = Neutral images_Control)Later trials,ERP amplitudes (P1/EPN/LPC): Emotional images_Social > Emotional images_Control > Neutral images_Social > Neutral images_ControlSample size:Maximum sample size of N = 95 based on a power analysis to give a power of 95% to significantly detect the smallest meaningful difference between cue conditions. Pre-defined inspection points for interim analysis N = 60N = 80 Stopping point N = 95An appropriate Type 1 correction will be applied as per the number of peeks into the data (for e.g., Pocock correction).We will also balance the biological sex of recruited participants.Inclusion criteria:Age between 18 to 35 years, German or English language proficiency to understand written instructions during the experiment, no history of diagnosed neuro-psychiatric disorders and normal or corrected-to-normal vision with lenses or glasses.Exclusion criteria:We will exclude participants having less than 30 valid EEG trials per combination of valence and cue condition in each of the Valence Classification and the Old/New Classification tasks. We will determine the rejection rate (EEG artifact-related dropouts) after collecting data from 20 participants. If more than 6 participants have to be excluded due to poor data quality, we will pause the data collection and reconsider the experimental design – remove some levels of the factors from the design.We will further exclude trials from behavioral analyses, where no response is given within the predefined time window of 2 s, as well as outliers (response time deviating 3 SDs from the mean response time for a combination of valence and cue condition per participant).If the criteria for the successful induction of for perceptual uncertainty are not met (Lower accuracy and slower response times in the control compared to the social cue condition in the first 2 blocks or the performance indicates a ceiling effect for the control cue condition), we will exclude the participant’s dataset from the analysis.Any termination of an experimental session will result in the complete exclusion of that participant’s data. Hypothesis 1. Generalized Linear Mixed Model (GLMM) with a binomial error structure and a logit link function with subject, image ID and image ID nested within subject as the grouping factors for the random effects.Accuracy ~ Valence * Cue Condition * Trials + Image Complexity + (1 + ValenceCue Condition* Trials + Image Complexity | subject) + (1 + Cue Condition* Trials | image ID) + (1 + Trials | image in subject)* Hypothesis 2. and 3. Linear Mixed Model (LMM), with subjects, image ID and image ID nested within subject as the grouping factors for the random effects.Dependent Variable (RT/ERPs: P1,*** EPN***,*** LPC)*** ~ Valence * Cue Condition * Trials + Image Complexity + (1 + ValenceCue Condition* Trials + Image Complexity | subject) + (1 + Cue Condition* Trials | image ID) + (1 + Trials | image in subject)For the ERP data, after assessing linearity, if the LMMs would not be the best fit, then we will consider non-linear models to fit the data. Model Fitting To keep the probability of Type 1 error at the nominal level of 0.05 and avoid an overconfident model, we will include all theoretically identifiable random slope components and correlations between random intercept and slope into the model. However, we will exclude the correlations from the model if and when they are unidentifiable (recognizable by the majority of the absolute correlation parameters being close to 1). Additionally, if we encounter convergence issues with our models, we will initially start by changing the optimizers used to fit the model and eventually simplify the models, i.e., in a first step drop the random slope term for the interaction and, if convergence issues persist, drop random slope parameters whose contributions are estimated to be very low.Inference - We will use a full versus null model comparison using a likelihood ratio test, where the full model includes both the fixed effects and the random effects and the null model that lacks all the fixed effects of interest, but retaining the control variables and the random effects to be controlled for. For assessing the significance of individual effects in the LMMs, we will use Satterthwaite approximation using the function lmer of the package lmerTest in R and a model fitted with restricted maximum likelihood. In case of GLMMs, we will assess the contribution of individual predictors using the likelihood ratio tests with the drop1 method (test = Chisq). We will obtain 95% confidence intervals of the model estimates by parametric bootstrapping. Additionally, to interpret null findings, we will use a frequentist equivalence testing with upper and lower bounds as the smallest meaningful effect of 0.2. Null Hypothesis – The emotional salience from the face expression does not transfer to the target images during the course of the task, hence showing no significant differences between the associated cue conditions (social/control) in both task performance and neural processing. Hence the null hypothesis assumes no effects of emotional salience transfer via associative processes.Performance measures – Accuracy and RTDuring the initial part of the task, we expect better performance (accuracy and RTs) for the social cue condition irrespective of valence indicating the use of facial expressions in accurate and faster classification of images. However, we expect this effect to evolve across trials into an interaction of valence and cue condition. We expect that there will be an overall improvement in performance across trials due to perceptual learning and this effect will be greater for emotional images. Further, this difference will be greater for emotional images that are associated with facial expressions due to the additional emotional salience from the facial expressions.ERP measures – P1, EPN, LPCWe expect the ERP to initially show valence effects which eventually interacts with cue conditions across trials resulting in enlarged amplitudes for emotional images associated with facial expressions. The interaction of valence and the cue condition evolving towards the later trials would indicate the transfer of emotional salience from face expression to target images as a result of social learning.Other Potential Outcome:If the three-way interaction between valence, cue condition, and trials does not show significance for both the behavioral performance and the ERP amplitudes, but there is a significant two-way interaction between valence and cue condition, we will conclude that the transfer of emotional salience across trials has a negligible effect that cannot be detected by the current design. This would also mean that emotional salience transfer from the facial expressions to the target images takes place rather early in the task, which then persists throughout the task, being much greater for emotional images as compared to neutral images (resulting in the interaction of cue condition and valence). Research Question 2. If the expected emotional salience transfer from the facial expressions to the target images occurred during the learning session, would the acquired and integrated emotional salience be preserved in the absence of the facial expressions, and what are the underlying neurocognitive modulations toward recognition memory of the target images in the test session?Null Hypothesis (H0) for Behavioral Performance in the Old/New Classification Task and ERP amplitudes: The recognition memory in task performance and neural difference in terms of ERP between the social and the control cue condition will not significantly differ, and there will be only effects of emotional compared to the neutral images. In addition, we will see the typical Old/New effects in the respective ERP components (see Hypothesis 7).H0: Accuracy/ERP amplitudes: (Old_Emotional images_Social = = Old_Emotional images_Control) > Old_Neutral images_ Social = = Old_Neutral images_ControlH0: RTs**: (Old_Emotional images_Social = = Old_Emotional images_Control) < (Old_Neutral images_ Social = = Old_Neutral images_Control) Hypothesis 4: Old/New Classification Accuracy Accuracy: Old_Emotional images_Social > Old_Emotional images_Control > Old_Neutral images_ Social > Old_Neutral images_Control Hypothesis 5: Old/New Classification RTs RTs: Old_Emotional images_Social < Old_Emotional images_Control < Old_Neutral images_ Social < Old_Neutral images_ControlHypothesis 6: ERP amplitudes (P1,** EPN**,** LPC)ERP amplitudes (P1, EPN**,** LPC): Old_Emotional images_Social > Old_Emotional images_Control > Old_Neutral images_ Social cue > Old_Neutral images_ControlHypothesis 7: ERP amplitudes (P300**,** LPC) – Old/New effectsERP amplitudes (P300**,** LPC)**: Old_Emotional images_Control > New_Emotional images > Old_Neutral images_Control > New_Emotional imagesSample size, inclusion and exclusion criteria will be the same as that of research question 1, since it is a within-subject design and the same set of participants will perform the tasks on both days. Hypothesis 4. Generalized Linear Mixed Model (GLMM) with a binomial error structure and a logit link function with subject, image ID and image ID nested within subject as the grouping factors for the random effects.Accuracy ~ Valence * Cue Condition * Trials + Image Complexity + (1 + ValenceCue Condition Trials + Image Complexity | subject) + (1 + Cue Condition Trials | image ID) + (1 + Trials | image in subject)Hypothesis 5,* 6 and 7.Linear Mixed Model (LMM), with subjects, image ID and image ID nested within subject as the grouping factors for the random effects.Dependent Variable (RT/ERPs: P1,* EPN*,*** LPC***,*** P300)*** ~ Valence * Cue Condition * Trials + Image Complexity + (1 + ValenceCue Condition* Trials + Image Complexity | subject) + (1 + Cue Condition* Trials | image ID) + (1 + Trials | image in subject)**For the ERP data, after assessing linearity, if the LMMs would not be the best fit, then we will use a non-linear model to fit the data. Model Fitting To keep the probability of Type 1 error at the nominal level of 0.05 and avoid an overconfident model, we will include all theoretically identifiable random slope components and correlations between random intercept and slope into the model. However, we will exclude the correlations from the model if and when they are unidentifiable (recognizable by the majority of the absolute correlation parameters being close to 1). Additionally, if we encounter convergence issues with our models, we will initially start by changing the optimizers used to fit the model and eventually simplify the models, i.e., in a first step drop the random slope term for the interaction and, if convergence issues persist, drop random slope parameters whose contributions are estimated to be very low.Inference - We will use a full versus null model comparison using a likelihood ratio test, where the full model includes both the fixed effects and the random effects and the null model that lacks all the fixed effects of interest, but retaining the random effects to be controlled for. For assessing the significance of individual effects in the LMMs, we will use Satterthwaite approximation using the function lmer of the package lmerTest in R and a model fitted with restricted maximum likelihood. In case of GLMMs, we will assess the contribution of individual predictors using the likelihood ratio tests with the drop1 method (test = Chisq). We will obtain 95% confidence intervals of the model estimates by parametric bootstrapping. Additionally, to interpret null findings, we will use a frequentist equivalence testing with upper and lower bounds as the smallest meaningful effect of 0.2.Null Hypothesis –Although the associations on day 1 were learned (reflected by both performance in the learning session, memory test immediately after the learning session and neural measures), the emotional salience from the facial expressions are not preserved in the target images in the absence of the facial expressions, indicating no significant differences between the old target images associated to different cue conditions (social/control). Hence the null hypothesis assumes no preservation of emotional salience in memory in the absence of the facial expressions during the test session on the following day.Performance measures – Accuracy and RTsIn the Old/New task, we expect interactions between valence and cue condition, with greater classification accuracy and faster RTs for the emotional images associated with facial expressions compared to emotional images in the control cue condition. This would indicate preservation of emotional salience from the facial expressions in the test phase which aids recognition memory.ERP measures – Salience effectsFor the ERPs (P1, EPN, LPC), the interaction of valence and cue condition would indicate the potency of emotional salience from both facial expressions and the target images. We expect greater effects for emotional images associated with facial expressions showing combined effects of inherent emotional salient and associated emotional salience. This neurocognitive modulation towards target images would suggest preservation of emotional salience in long-term memory even in the absence of facial expressions.ERP measures – Old/New effects checkWe expect the ERPs (P300, LPC) to show Old/New effects, with enhanced positive-going amplitudes for the Old emotional target images (only control condition) compared to new emotional images. This result would be a check for typical Old/New effects (greater positive-going amplitude of the ERPs for the old emotional images compared to the new emotional images), and would indicate that target images were attended to and processed on day 1, considering the task difficulty of the paradigm, where images are viewed under perceptual uncertainty.
Research question 1
Does the emotional salience of facial expressions integrate with that of target images under perceptual uncertainty during the learning session, and if so, what are the neuro-cognitive mechanisms underlying this emotional salience transfer?
For the learning session, we expected our hypothesized effects (as described below) to emerge through learning across trials. Accordingly, we tested all effects of our experimental manipulation as an interaction with trials. To specify the direction of the interaction, we divided our hypothesis into earlier and later trials, although the statistical models treated trials as a continuous covariate. As we did not expect any differences in the direction of the hypothesized effects, positive and negative valence were grouped together under the category “emotional images” to simplify the effects’ specification. The experimentally manipulated cue conditions conveying facial expressions were labelled as ‘social’ and those conveying scrambled face as ‘control’. The trial numbers were determined separately for each target image in their order of appearance and not for each target image valence-cue condition combination, since learning occurred independently for each target image.
Valence classification task
Behavioral performance measures
Null Hypothesis (H0): behavioral performance measures (valence classification task)
The valence classification accuracy and the response time would not differ significantly between the social and the control cue condition across trials. However, we anticipated differences in both accuracy and response time for emotional compared to neutral images. That is, greater accuracy and faster response times for the emotional compared to neutral images irrespective of the associated cue condition (social/control).
Hypothesis 1: valence classification accuracy
At the beginning of the experiment, i.e., earlier trials, we expected an overall greater classification accuracy and faster response times in the social cue condition compared to the control cue condition, with comparable performance measures for emotional and neutral images, as the uncertainty induced by brief stimulus durations necessitates the use of facial expressions in the valence classification.
Across trials, we expected an enhancement in overall performance as a result of perceptual learning, with greater performance in emotional compared to neutral images because of their inherent emotional salience. Thus, we expected increasing effects of the target image valence to reveal in the behavioral performance measures over the time course of the learning session, resulting in an interaction of the valence of the target image (positive/negative/neutral) and cue condition (social/control). We expected emotional images in the social cue condition to show greater performance gains due to the expected integration of emotional salience from the facial expressions with the inherent emotional salience of the emotional images. This would be followed by the emotional images in the control cue condition due to the presence of inherent emotional salience.
For earlier trials, we expect:
(Emotional images_Social = = Neutral images_Social) > (Emotional images_Control = = Neutral images_Control).
For later trials, we expect:
Emotional images_Social > Emotional images_Control > Neutral images_Social > Neutral images_Control.
Hypothesis 2: response times (RTs)
For earlier trials, we expect:
(Emotional images_Social = = Neutral images_Social) < (Emotional images_Control = = Neutral images_Control).
For later trials, we expect:
Emotional images_Social < Emotional images_Control < Neutral images_Social < Neutral images_Control.
Null hypothesis (H0) : ERP amplitudes for components P1, EPN, LPC
The neural difference in terms of ERP amplitudes would not significantly differ between the social and the control cue condition. We expected a difference only between the emotional and neutral images. That is, the emotional compared to neutral images will elicit enhanced neural processing, resulting in greater ERP amplitudes irrespective of the associated cue condition (social/control).
Hypothesis 3: ERP amplitudes for components P1, EPN, LPC
At the beginning of the experiment, i.e., earlier trials, we expected comparable ERP amplitudes between the social and control cue condition since learning of the associations is only expected to occur through the course of the learning session. In contrast to the behavioral performance measures (see Hypothesis 1–2), the effect of the associated cue condition (social/control) would not be immediately apparent in the ERP components, as they are time-locked to the target image presentation, while the associated cue condition (social/control) always succeeds the target image within the task.
Thus, across trials during the learning session, we expected effects of cue condition (social/control) to unfold over time, resulting in an interaction with the valence of the target images (positive/negative/neutral). Specifically, we expected enhanced P1 and EPN amplitudes towards emotional images in the social cue condition, compared to the control cue condition. Such results would indicate increased selective attention driven by the combined influence of emotional salience from the facial expressions and the inherent emotional salience of the emotional images. The LPC has been assumed to reflect sustained elaborative stimulus processing and increased memory encoding of emotional stimuli. We thus expected enhanced LPC amplitudes to emotional images associated with facial expressions.
For earlier trials, we expect:
(Emotional images_Social = = Emotional images_ Control) > (Neutral images_Social = = Neutral images_Control).
For later trials, we expect:
Emotional images_Social > Emotional images_Control > Neutral images_Social > Neutral images_Control.
Research question 2
If the emotional salience transfers from facial expressions to the target images during the learning session, would the acquired and integrated emotional salience be preserved in the absence of the facial expressions, and what underlying neurocognitive mechanisms underlie the recognition memory of the target images during the test session?
Since the new target images only varied in their inherent emotional salience and had no associated emotional salience from the facial expressions, we only compared cue conditions (social/control) of the old target images to assess the preservation of emotional salience through prior associative learning. In addition, we compared the new target images to the old target images in the control cue condition to verify typical Old/New effects in the ERP components and ensure that the images were attended to. Specifically, we hypothesized that the old target images with emotional salience will elicit more positive-going ERP amplitudes compared to the new target images with emotional salience.
Similar to the learning session, we tested for an interaction of the valence of the target images (positive/negative/neutral), cue condition (social/control), and trial in our statistical models. We expected this interaction to capture potential effects of memory extinction over time, if any. However, we only state hypothesis that are relevant with the respect to the research question. Once again, for the hypotheses, the positive and negative valence will be grouped together under the category “emotional images” to simplify the effects’ specification.
Old/new classification task
Behavioral performance measures
Null Hypothesis (H0): behavioral performance measures (old/new classification task)
The recognition memory and response time between the social and the control cue condition would not significantly differ and we only expected to see differences between the emotional and neutral images, where the emotional images are remembered better and classified faster compared to the neutral images.
Hypothesis 4: old/new classification accuracy
For the Old/New classification accuracy, we hypothesized an interaction between the valence of the target image (positive/negative/neutral) and cue condition (social/control) due to the expected transfer of emotional salience from the facial expressions. This would result in combined effects of emotional salience from the target images and the facial expressions, reflected in better recognition memory for the old emotional images in the social compared to the control cue condition. Similarly, we expected faster responses to the old emotional images in the social compared to the control cue condition.
Old_Emotional images_Social > Old_Emotional images_Control > Old_Neutral images_ Social > Old_Neutral images_Control.
Hypothesis 5: response times (RTs)
Old_Emotional images_Social < Old_Emotional images_Control < Old_Neutral images_ Social < Old_Neutral images_Control.
ERP measures
Null Hypothesis (H0): ERP amplitudes for components P1, EPN, LPC – Salience effects
The difference in neural activity, indicated by ERP amplitudes, between the social and control cue conditions would not significantly differ, and we will only see differences between emotional compared to neutral images. Additionally, we expected to observe typical Old/New memory effects in the respective ERP components, with larger positive-going amplitudes for the old emotional images (only the control cue condition) than new emotional images.
Hypothesis 6: ERP amplitudes for components P1, EPN, LPC – Salience effects
We expected an interaction of the valence of the target images (positive/negative/neutral) and cue condition (social/control), indicating the integrated effects of emotional salience of the facial expressions and the emotional images through prior associations.
In the social cue condition, old emotional images should gain additional emotional salience due to the integration of the emotional salience from the facial expressions, resulting in an enhanced selective attention and perceptual encoding. This should be reflected in larger P1 and EPN amplitudes for images that had been associated with facial expressions during learning. Similarly, we expected enhanced amplitudes of the LPC amplitudes for emotional old target images associated with facial expressions because they are strongly encoded in memory, leading to better recognition and more allocation of processing resources.
Old_Emotional images_Social > Old_Emotional images_Control > Old_Neutral images_ Social > Old_Neutral images_Control.
Hypothesis 7: ERP amplitudes for components P300, LPC – old/new effects
In line with previous studies, we expected typical Old/New ERP effects on the P300 and LPC components. Thus, we expected enhanced positive-going amplitudes to old emotional images (only the control cue condition) compared to new emotional images.
ERP amplitudes (P300, LPC):
Old_Emotional images_Control > New_Emotional images > Old_Neutral images_Control > New_Neutral image.
Methods
Ethics information
The study was conducted in accordance with the Declaration of Helsinki, with the approval of the local Ethics committee of the Institute of Psychology at the University of Göttingen. Participants received full information about the procedure and only participated in the experiment after giving their informed consent.
Design
Day1: Part A – learning session (valence classification task)
The experiment was divided into two sessions conducted on two separate and consecutive days. A self-written script in standard python (v.3.8) utilizing PsychoPy^72^ was used for stimulus presentation during testing.
Fig. 1. Day 1 and Day 2 experimental design. (a). Trial scheme of the Valence Classification Task with detailed time sequence. (b). Trial scheme of the Old/New Classification Task with detailed time sequence.
The valence classification task consisted of a 2 (cue condition: social/control) x 3 (target image valence: positive/negative/neutral) within-subject design. Participants classified the valence of each of the images as either positive, negative, or neutral.
Before the start of the experiment, the participants were prepared for the EEG recording with a 64-electrode EEG cap. The EEG was recorded with 64 active electrodes (AgAgCl) mounted in an electrode cap (Easy Cap™) according to the extended 10–20 system^73^. For the reference, the common mode sense (CMS) active electrode and for the ground, the driven right leg (DLR) passive electrode was used. The scalp voltage signals were amplified by a BiosemiActiveTwo AD-Box (24 bits; band-pass filter 0.16–100 Hz) recorded by the recording software ActiView at a sampling rate of 512 Hz. Additionally, two external electrodes, one on each for the left and right mastoids were used. To record the electrooculography (EOG), the 4 external electrodes that were placed inferior and laterally to the left and right eyes were used.
The participants were seated approximately 55 cm away from the stimulus monitor and instructed to rest their chin on an adjustable chin rest. Electrode offsets were checked to keep them below a threshold of 土 20 mV. Participants then read the instructions at their own pace and completed 10 practice trials. The practice trials contained unique images that did not appear again in the main experiment.
Each trial (see Fig. 1a.) started with a centrally presented forward mask for 500 ms with a randomly sampled uniform distribution of jitter ranging between 土 200 ms. The forward mask was centrally presented at a visual angle of approximately 13 × 13 degrees, followed by the target images presented centrally at a visual angle of approximately 2.7 × 2.7 degrees with a 100 × 100 pixel resolution. Immediately following the forward mask, the target image stimulus was presented for a brief duration of 27 ms. This was followed by a blank screen lasting 973 ms, which allowed for the processing of the briefly presented target image. A fixation cross appeared for 200 ms with a jitter of randomly sampled uniform distribution, ranging between 土 200 ms, to reorient attention to the center of the screen. Following this, the face stimulus (face expression or scrambled face) was presented for 500 ms and a second fixation cross was immediately presented on the disappearance of the face stimulus until the participant’s responded by pressing one of the pre-assigned keys. The faces were presented at a visual angle of about 8 × 9 degrees with a resolution of 300 × 400 pixels. The inter-trial interval lasted for 1000 ms with a jitter of randomly sampled uniform distribution, ranging between 土 200 ms.
The valence classification task contained 10 blocks of 42 target images each (14 per valence). The target images were associated with either social cues or scrambled faces with equal probability and the associations remained consistent throughout the task. In the social cue condition, the target images were always congruently associated to a facial expression (positive image – happy facial expression; negative image – sad facial expression; neutral image – neutral facial expression). Each of the six possible combinations of the target image valence and its associated cue condition (social/control) were presented twice within each block. This resulted in 84 trials per block and a total of 140 trials for each of the six combinations across all blocks. There were short breaks between the blocks.
Day 1: Part B – memory test (Immediately after the Learning session)
Following the learning session, participants performed a memory test, which allowed us to examine their acquisition of associations between the target images and the facial expressions. The test consisted of two blocks – the first block assessed recall and the second block assessed recognition memory. The instructions for each of the two blocks of the memory test were presented separately right before the respective blocks. In each block, every target image seen during the learning session were presented once, resulting in 42 trials each. There were 84 trials in total in the memory test.
In the first block (recall), all the target images were presented under perceptual uncertainty (as described in the methods for Day1: Part A – Learning session (Valence Classification Task), without the social or control cue condition. Participants indicated the correct facial expression that was paired with the target image in the learning session by keypresses. To indicate a scrambled face, participants clicked on the ‘SPACE’ button. The trial scheme for the first block is identical to the Valence Classification Task, except for the second fixation cross and the face.
In the second block (recognition), the target images were presented together with faces (incl. facial expressions and scrambled faces) in identical or different combinations compared to the learning session. Participants indicated if the combination is the same as witnessed during the learning session or not. Half of the target images were presented with the same face as in the learning session and the other half with a different face. The trial scheme for the second block is identical to the Valence Classification Task.
In both blocks, participants were asked to respond as quickly and accurately as possible, within a time-window of 2 s.
Day 2: Test session (old/new classification task)
The Old/New classification task consisted of a 3 (social cue condition/control cue condition/new images) x 3 (target image valence: positive/negative/neutral) within-subject design. Participants were asked to classify the images as old (seen on the previous day) or new images.
Seating of the participant, technical setup, and preparation of the EEG recordings followed the same procedure as in the learning session. Again, participants were allowed to read the instructions at their own pace, followed by 10 practice trials of images that did not appear again in the main experiment.
Each trial (see Fig. 1b.) began with a centrally presented fixation cross for 500 ms with a jitter of randomly sampled uniform distribution with a range between 土 200 ms, followed by the image for a period of 1000 ms. The target images were presented centrally at a visual angle of approximately 5 × 3 and a resolution of 300 × 200 pixels. After the disappearance of the target image stimuli, another fixation cross was presented until the response of the participant. The inter-trial interval lasted for 1000 ms with a jitter of randomly sampled uniform distribution of a range with 土 200 ms.
There were 10 blocks of Old/New classifications, consisting of 42 old target images presented twice and 42 new target images presented once, resulting in a total of 126 trials per block. This resulted in a total of 140 trials per combination of target image valence (positive/negative/neutral) and cue condition (social/control) for the old set, and 140 trials for each of the target image valence of the new set.
Coding of trial number
For both Day 1 and Day 2 tasks, trial number was defined at the level of each individual image to reflect how many times that specific image had been encountered within the session. Because the sequenceof images was fully randomized within each block, the first time an image appeared it was coded as trial 1, the second time as trial 2, and so forth. In both the learning session (Day 1) and the test session (Day 2), each image was presented twice within each of the 10 blocks, resulting in a total of 20 presentations per image per session. Accordingly, trial number ranged from 1 to 20 for every image on both days. This image-specific trial index captures exposure-dependent changes in behavior and neural responses while remaining independent of the absolute trial position in the task sequence.
Stimuli
Images (see Table S1. in Supplementary Information 1) were selected from the IAPS database^74^. Our selected set followed the general pattern of higher mean arousal ratings for negative images in the database compared to positive ones. The chosen images were further restricted based on their mean complexity scores (as part of the IAPS database). Images with easier figure-ground discriminability (image complexity rating less than 2.5) were selected since ERPs to the early components might be influenced by image complexity^75^.
The selected images were normalized using an open-source MATLAB code^76^ to ensure they have the same mean luminance and root-mean-square (RMS) contrast. Facial expressions (happy/sad/neutral) from three male and female portrayers were randomly selected from the Radboud face database^77^.
Forward mask
A single forward mask was used throughout the Day 1 learning session and the immediately following Day 1 memory test, identical for all participants and trials. The mask was a photograph of a closed wooden door selected from a publicly available stock-image repository. To prevent low-level visual differences between mask and target images, the image was normalized to match the mean luminance and root-mean-square (RMS) contrast of the target stimuli. The mask contained no emotional content and was not unrelated to any of the affective categories under investigation.
The mask served a specific perceptual purpose. Unlike traditional masking paradigms that employ high-contrast noise masks to limit or block conscious stimulus access to the target, the present study did not aim to prevent stimulus awareness. Instead, the goal was to induce perceptual uncertainty about the valence of the briefly presented target image while preserving conscious perception of its visual content. A forward mask with coherent visual structure — but without emotional relevance — was therefore preferable to pixelated or noise-based masks, which would have interfered with stimulus detection or early perceptual processing.
Randomization
Two sets of images, each consisting of 42 images (14 negative, 14 positive and 14 neutral) were defined by random allocation of the images from the pre-selected pool for every participant. One of image sets was used in the Day 1 learning session and associated with either a social or a control cue condition. For the Day 2 test session, the second image set served as new images and was randomly shuffled along with the old images. For the learning session, every image was randomly assigned to either the social or control cue condition. The assignments remained constant for a participant throughout the session. Face stimuli was randomly selected from the pre-determined set of face stimuli for each participant, which ensured an equally likely occurrence of every combination of the biological sex of both the participant and the posers in the face stimuli (Female participant – Male face stimuli, Female participant – Female face stimuli, Male participant – Male face stimuli, Male participant – Female face stimuli).
For the second block of the memory test that immediately followed Day 1 learning session, the correct and incorrect associations of the target images-face stimuli was randomly assigned to each target image for each participant.
The assignment of response keys was also counterbalanced for the valence classification, memory test, and the old/new classification. The study did not involve any blinding; hence, data collection and analysis was not conducted blindly to the experiment’s cue conditions.
Participants
A total of N = 80 participants were recruited for the study (mean age: 23.33 ± 3.72 SD; 45 female, 34 male, 1 other; 72 right-handed, 6 left-handed, 2 ambidextrous) through online and offline advertisements, flyers, mailing lists, and social media postings. Participants were compensated at a rate of 8.50 EUR per hour of the experimental session or provided course credits. All participants were native German speakers with normal or corrected-to-normal vision and no self-reported history of neurological or psychiatric disorders. All were able to read and understand study instructions in German or English.
Sample size justification
We conducted simulation-based power analyses using expected P1 amplitude differences to determine the required sample size. Fixed-effect sizes for target-image valence (positive/negative/neutral), cue condition (social/control), and trial effects (and their interactions) were simulated in the range of 0.15–0.30, alongside random-effect standard deviations of 0.30–0.60. These ranges reflected both the novelty of the paradigm and the substantial inter-individual variability observed in previous associative-learning ERP studies.
Across 1,000 simulated linear mixed-effects models, we estimated the probability of detecting the preregistered three-way interaction (valence × cue condition × trial) at different sample sizes. Based on these simulations, we preregistered a target sample size of N = 95, corresponding to approximately 95% power to detect the smallest expected effect.
Because the simulated effect sizes were conservative and data collection was constrained by practical considerations, the preregistered analysis plan permitted up to three planned inspections of the accumulating data to determine whether data collection could be concluded earlier. Specifically, interim analysis were allowed at N = 60, N = 80, and, if necessary at the full target of N = 95. To control the inflation of Type I error associated with these three planned looks at the data, we applied a Pocock group-sequential correction, which yields a constant, more conservative significance threshold of α = 0.0221 for all preregistered hypothesis tests, irrespective of the point at which data collection was stopped.
Exclusion criteria and data checks
We recruited additional participants to account for potential exclusions due to excessive EEG artifacts, insufficient numbers of valid trials, or dropout between sessions. In line with our lab’s established protocols, participants were to be excluded if they contributed fewer than 30 valid EEG trials per combination of valence (positive/negative/neutral) and cue condition (social/control) in both the Valence Classification and Old/New Classification tasks. As preregistered, EEG data quality was evaluated after the first 20 datasets to determine whether the exclusion rate warranted adjustments to the study design. Specifically, if more than six participants had required exclusion due to poor data quality, data collection would have been paused and the task parameters reconsidered. In practice, no dataset met the exclusion criteria, and no design modifications were necessary.
From both the learning and the test session, trials with missing responses (i.e., no response within the predefined 2-second time window), and trials with response times ± 3 SDs of the participant-specific mean for each valence x cue condition were excluded from the behavioral analyses.
To verify that the experimental manipulation successfully induced perceptual uncertainty, we preregistered an evaluation of classification accuracy and response times during the first two blocks of the learning session. Lower accuracy and slower response times in the control cue condition relative to the social cue condition were taken as evidence of reliance on the social cue to resolve perceptual ambiguity. If this difference was absent but accuracy in both cue conditions remained at chance level (33.33%), the participant was retained under the assumption that implicit associative learning could still emerge. In contrast, a ceiling effect in both cue conditions – indicating failure of the perceptual-uncertainty manipulation – would lead to exclusion. Following preregistration, these criteria were to be reassessed after the first 20 datasets; if more than one quarter of participants required exclusion, data collection would be paused and the task design reconsidered. As reported in the Results, no participant met these exclusion criteria.
For the memory test administered immediately after the learning session, we assessed each participant’s accuracy and response times. Accuracy above chance in block 1 (25%; four response options: happy, sad, neutral, scrambled) and block 2 (50%; two response options: same, different), together with faster responses for correct than incorrect associations, was taken as evidence of successful retrieval of the learned contingencies. The preregistration specified that participants who failed to meet these criteria would be retained only following a sensitivity analysis comparing results with and without their data. In practice, all participants met the memory test criteria, and no sensitivity analyses or exclusions were required.
Any premature termination of an experimental session resulted in the complete exclusion of that participant’s dataset, and additional participants were recruited as replacements.
Analysis plan
EEG preprocessing pipeline
The preprocessing pipeline used followed the standardized procedures used in our lab. The EEG data was preprocessed using the EEGLAB (version 2022.0)^78^, ERPLAB (version 9.10)^79^, and BioSig (version 3.8.1)^80^ toolboxes in MATLAB (R2018a). The signal was referenced to averaged mastoids and was processed through an offline high-pass filter 0.1 Hz. Further, a 50 Hz notch filter was also applied to filter out line noise. The data was segmented from − 200 to 1000 ms relative to the onset of the target image and corrected to a 200 ms pre-stimulus baseline. Independent Component Analysis (ICA) will be used to remove artifacts such as eye blinks and muscle or channel noise. The data was inspected for any channels with poor signal quality and any removed channels was interpolated using the spherical interpolation algorithm. Then, noisy trials were rejected if voltage values exceed the thresholds of −100 µV or + 100 µV within the time window of −200 ms to 700 ms, or if the slope of the voltage trend reaches 100 µV or more within the window of interest for component analysis. Data was defined as statistically improbable and therefore rejected, if a trial contains values exceeding 5 SD of the mean probability distribution (at the channel level as well as for all channels). Finally, the signal was re-referenced to the average of all channels and was used for further statistical analysis.
Based on previous studies, we quantified mean ERP amplitudes at the following time windows (in milliseconds (ms)) and regions of interests (ROIs):
P1: 80–120 ms; occipital cluster: O1, O2, Oz.
EPN: 250–300 ms; occipital-temporal cluster: O1, O2, P7, P8, P9, P10, PO7, PO8, PO3, PO4.
LPC: 400–600 ms; centro-parietal cluster: CP1, CPz, CP2, P3, Pz, P4, PO3, POz, PO4.
P300: 300–400 ms; parietal cluster: Pz, P1*, P2*, PO3, PO4.
*In the stage 1 Registered Report the regions of interest were incorrectly entered as PO1 and PO2.
Given the considerable task difficulty of our paradigm, we modified the time-windows and ROIs after visualizing the actual scalp distribution and latency of the ERP of interest (reported in the Results section).
Statistical analysis
All the Statistical analysis (preregistered and exploratory) was performed using R statistical software (4.2.0 or higher, R Core Team, Vienna, Austria) and RStudio (version 2022.02.2, RStudio, Boston, MA, USA).
To investigate the probability of correctly classifying an image into the predefined valence category and Old/New categories, we used a Generalized Linear Mixed Model (GLMM) with a binomial error structure and a logit link function with subject, image ID, and image ID nested within subject as grouping factors for the random effects. For the response times and ERPs (P1, EPN, P300 and LPC amplitudes) measured in both tasks, we used Linear Mixed Models (LMMs) with subject, image ID, and image ID nested within subject as the grouping factors for the random effects.
Fixed and random effects
All the models used to test the specified hypotheses had an interaction of valence (positive/negative/neutral), cue condition (social/control), and trials. Trials were included as a covariate in the fixed effect parts of the model as an interaction with the other factors to assess the effects of learning in the learning session and extinction in the test session. In addition, to control for the effect of image complexity, the complexity ratings obtained from the IAPS database were added to the fixed effects part of model. All the models had ‘Subject ID’, ‘Image ID’ and Image nested within Subject (‘Images in Subjects’) as grouping factors for the random effects. The ‘Images in Subjects’ grouping factor was created by combining the ‘Subject ID’ and the ‘Image ID’. The specified random effects avoid pseudo-replication and control for variation between subjects, images and subject-specific variation of image effects with regard to the average response respectively. We included all theoretically identifiable random slopes based on at least 3 or more unique observations of the covariate per subject or at least 2 or more observations per level of the factor per subject. Thus, for ‘subject ID’, we included the interaction of valence (positive/negative/neutral), cue condition (social/control), and trial and the Image Complexity as they each meet the criteria for the specification of random slopes. For the ‘image ID’ since each image uniquely belongs to only one level of the factor valence and has one value for the image complexity rating, we could only include the interaction of valence (positive/negative/neutral) and cue condition (social/control). Lastly, for the grouping factor ‘Image in subject’, each image nested within a subject belongs to only level of the factor valence (positive/negative/neutral) and cue condition (social/control). Thus, we only included the random slopes of the trials for ‘Image in subject’.
The models fitted were:
GLMM.
- Accuracy (Valence/Old New Classification Task) ~ Valence * Cue Condition * Trials + Image Complexity + (1 + Valence Cue Condition* Trials + Image Complexity | subject) + (1 + Cue Condition * Trials |Image ID) + (1 + Trials |Image in subject)*.
LMM.
- ERP amplitudes/Response Times (Valence/Old New Classification Task) ~ Valence * Cue Condition * Trials + Image Complexity + (1 + Valence * Cue Condition * Trials + Image Complexity |subject) + (1 + Cue Condition * Trials |Image ID) + (1 + Trials |Image in subject).
Model fitting
To maintain the nominal Type 1 error rate of 0.05 and avoid overconfident model estimates, we initially specified the maximal random-effects structure permitted by the design, including all theoretically identifiable random slopes and correlations between random intercepts and slopes. Correlations were removed only when unidentifiable, indicated by the majority of the absolute correlation parameters being close to 1. As preregistered, we implemented a systematic procedure to address potential convergence issues in our mixed-effects models. First, if a model failed to converge, we planned to refit it using alternative optimizers. For the LMMs, the default optimizer (“nloptwrap”) would be replaced with “bobyqa,” and for the GLMMs the default sequence (“bobyqa” and “Nelder_Mead”) would be supplemented by refitting with “nloptwrap.” If convergence problems persisted, we planned to use the allFit() function in R to evaluate a broad set of optimizers and identify the one producing the highest likelihood. Second, we planned to simplify the models, i.e., in a first step drop the random slope term for the interaction and, if convergence issues persist, drop random slope parameters whose contributions are estimated to be very small.
For ERP analyses, we evaluated residuals plotted against fitted values and residuals plotted against trial number to assess linearity. If evidence of non-linearity had emerged, we would have applied transformations to the response variable or the trial predictor, and, if required, considered fitting non-linear models.
In practice, none of the models exhibited persistent convergence problems, and no transformations or non-linear models were required.
Model validation
For all models, the independence of the residuals and the lack of highly influential outliers was assessed. In addition, for the LMM models, we checked the assumptions of normality and homogeneity of the residuals by means of a qq-plot and residual plotted against fitted values and applied data transformations if assumptions were violated.
We assessed the model stability using the DFBeta method by excluding levels of the grouping factor one at a time to see the effect on the model coefficients. We used the cut off 土 2/sqrt(N), where N is the sample size. If DFBETA values for any predictor had exceeded the prespecified cutoff, we planned to report that the corresponding model estimates were unstable with respect to that predictor and to interpret those effects with appropriate caution. In practice, no such instances were detected that required implementing these additional interpretive constraints.
Transformations
The models had 2 fixed effect categorical variables that were dummy coded with “neutral valence” and “control cue condition” as the reference levels for the target image valence (positive/negative/neutral) and cue condition (social/control), respectively. Additionally, the random slopes included in the models were dummy coded and mean centered. The covariates trials and image complexity were z-transformed to make the model estimates more interpretable^81^ and to facilitate model convergence.
The valence and old/new classification accuracies were transformed into the odds of correct or incorrect valence classification. We planned to log-transform the response time data if assumptions of the LMM were violated. In practice, no such instances were detected and did not require data transformation.
Inference criteria
To avoid multiple testing, we conducted a full versus null model comparison using a likelihood ratio test, where the full model included both the fixed effects and the random effects, and the null model omitted all fixed effects of interest^82^ but retained the variables and random effects to be controlled for. The fixed effects of interest were, valence (positive/negative/neutral), cue condition (social/control), and trials, and any of their interactions. To assess the significance of individual effects in the LMMs, we used the Satterthwaite approximation^83^ using the function lmer of the package lmerTest in R and a model fitted with restricted maximum likelihood. In the case of GLMMs, we assessed the contribution of each predictor using likelihood ratio tests with the drop1 method (test = Chisq).
If the full-null model comparison revealed significance, but the three-way interaction between valence (positive/negative/neutral), cue condition (social/control), trial did not, we dropped the three-way interaction, subsequently any potential non-significant two-way interactions, to obtain reliable estimates that are unconditional on particular values of other predictors with which they interact in the model.
Parametric bootstrapping was used to obtain 95% confidence intervals of the model estimates. Additionally, to interpret null findings, we used frequentist equivalence test with upper and lower bounds as the smallest meaningful effect of ± 0.2.
Exploratory analyses
In addition to the preregistered analyses, we conducted exploratory analyses to examine how participants’ subjective interpretations of the target images influenced behavioral and neural responses. The task was designed to induce perceptual uncertainty, and participants did not always classify the target images in accordance with their normative valence. Incorporating classification accuracy as an additional factor therefore allowed us to capture this subjective variability and assess whether reliance on social cues differed as a function of an individual’s moment-to-moment uncertainty.
Coding classification accuracy during the day 1 valence classification task (learning session)
For each trial in the learning session, the participant’s response was coded as accurate or inaccurate depending on whether it matched the target image’s normative valence. This variable was used as an index of subjective perceptual uncertainty. Accurate classifications were interpreted as trials in which the perceptual evidence was sufficiently clear for the participant’s impression to align with the normative valence. In contrast, inaccurate classifications indicated trials in which the perceptual signal was weaker or more ambiguous, indicating an overall greater uncertainty. In this way, the classification accuracy served as a trial-wise proxy for the level of perceptual uncertainty experienced by the participant.
A small proportion of trials (6.3%; 3,480 trials) involved directly opposing classifications, such as labeling a negative image as positive or a positive image as negative. These trials were excluded because such responses do not reflect the expected pattern of uncertainty under brief stimulus exposure. When perceptual information is insufficient, participants typically defaulted to the “neutral” classification rather than the opposite emotional pole. Neutral images are conceptually different in this respect: because they lie at the midpoint of the valence dimension, both “positive” and “negative” classifications represent plausible subjective interpretations under uncertainty. For this reason, opposite-category responses were retained for neutral images, but not for positive or negative ones. This approach preserves the conceptual meaning of the classification accuracy variable as a measure of subjective uncertainty.
Coding classification accuracy during the day 2 old/new (test session)
Participants did not provide valence judgments during the test session. To incorporate subjective perceptual uncertainty into the test session analyses, each target image was assigned an accuracy label (accurate/inaccurate) based on its classification history during learning. For each target image, we identified the most frequently selected classification across its final 18 of 20 learning session presentations; the first two presentations were excluded to avoid early noise while participants were adjusting to the task. If the modal response across these 18 trials matched the image’s normative valence on at least 14 occasions, the image was labeled as accurately classified; otherwise, it was labeled inaccurately classified. This procedure yielded a stable, image-specific estimate of how consistently participants perceived the image’s valence during learning. Using this aggregated uncertainty index in the test-session models allowed us to examine whether variability in subjective perceptual uncertainty during learning predicted behavioral and neural responses when explicit valence judgments were no longer required.
Classification accuracy was coded at the trial level during the learning session because participants provided a valence judgment on every trial, and each judgment reflected their immediate perceptual impression under brief and masked viewing conditions. Moreover, because the social cue followed the target image during learning, individual trial responses could also be influenced by the cue, making a trial-wise measure appropriate for capturing moment-to-moment subjective uncertainty.
In contrast, participants did not provide valence judgments during the test session. For this reason, classification accuracy had to be computed at the image level, based on the consistency of responses during the learning session. This image-level accuracy measure provides a stable estimate of how reliably each image was perceived during learning and enables the incorporation of subjective uncertainty into the test-session analyses despite the absence of trial-wise valence judgments.
Statistical analysis for exploratory analyses
Exploratory analyses examined whether participants’ subjective valence judgments, indexed via classification accuracy, modulated behavioral and ERP responses beyond the preregistered model structure. Because these analyses were not part of the preregistered hypotheses, they were conducted on aggregated data to obtain stable estimates of the effects of subjective uncertainty. For each participant, accuracy, response times and ERP amplitudes were averaged within each combination of cue condition (social/control), valence (positive/negative/neutral), and classification accuracy (accurate/inaccurate).
To investigate the probability of correctly classifying an image into the predefined Old/New categories (Day2), we used a GLMM with a binomial error structure and a logit link function with subject as grouping factor for the random effects. For the response times (Day1) and ERPs (P1, EPN LPC for Day 1 and EPN LPC for Day 2) measured in both tasks, we used LMMs with subjects as the grouping factor for the random effects.
Fixed and random effects
All the models used for the exploratory analyses had an interaction of valence (positive/negative/neutral), cue condition (social/control), and classification accuracy (accurate/inaccurate). In addition, to control for the effect of image complexity, the complexity ratings obtained from the IAPS database were added to the fixed effects part of model. All the models had ‘Subject ID’ as grouping factor for the random effects. We included all theoretically identifiable random slopes based on at least 3 or more unique observations of the covariate per subject or at least 2 or more observations per level of the factor per subject. Thus, for ‘subject ID’, we included the interaction of valence (positive/negative/neutral), cue condition (social/control), and classification accuracy and the Image Complexity as they each meet the criteria for the specification of random slopes.
The models fitted were:
GLMM.
- Accuracy (Valence/Old New Classification Task) ~ Valence * Cue Condition * Classification Accuracy + Image Complexity + (1 + Valence Cue Condition* Classification Accuracy + Image Complexity | subject)*.
LMM -.
- ERP amplitudes/Response Times (Valence/Old New Classification Task) ~ Valence * Condition * Classification Accuracy + Image Complexity + (1 + Valence * Cue Condition * Classification Accuracy + Image Complexity |subject).
All the model validations, data transformations and inference criteria were identical to the preregistered analysis plans.
Results
Based on the estimated sample size from the preregistered power analysis, data collection was planned with up to three possible inspection points: an initial interim analysis at N = 60, a second look at N = 80, and a final look at the full target sample of N = 95. To appropriately control for the inflation of Type I error associated with these three planned looks at the data, we applied a Pocock group-sequential correction. This yields a constant, more conservative significance threshold of α = 0.0221 for all preregistered analyses, regardless of the point at which data collection was concluded.
Further, to control for the inflation of Type I error arising from conducting multiple preregistered and exploratory analyses, we applied a False Discovery Rate (FDR) correction to the p-values obtained from each family of related models. For each family of tests (Day 1 behavioral, Day 1 ERP, Day 2 behavioral, Day 2 ERP, exploratory Day 1 behavioral, exploratory Day 1 ERP, exploratory Day 2 behavioral, exploratory Day 2 ERP), we aggregated the p-values for all fixed effects within that family and applied the Benjamini–Hochberg procedure.
All LMMs were fit using the “bobyqa” optimizer, and the GLMMs used “nloptwrap”.
Equivalence tests were conducted to evaluate whether non-significant effects could be interpreted as statistically equivalent to a predefined smallest effect size of interest (SESOI). In the preregistered analysis plan, the SESOI was set to ± 0.2 standardized units, corresponding to the canonical definition of a “small” effect size in conventional behavioral science. This standardized SESOI was converted into microvolts for each ERP component by multiplying ± 0.2 by the empirical single-trial standard deviation of that component. This conversion ensured that the equivalence bounds were expressed on the natural scale of the dependent variable and were directly interpretable within the LMM framework used for the analyses.
In addition to the preregistered SESOI, we also examined a narrower SESOI of ± 0.1 standardized units in exploratory analyses. Early visual ERP components such as the P1 and EPN typically show modulations that fall well below ± 0.2 SD of single-trial variability, and align with prior studies^84^ that report confidence-intervals consistent with approximately ± 0.1 SD. Thus, the ± 0.1 SESOI represents a more conservative and neurophysiologically realistic bound for the magnitude of ERP effects that would be considered practically relevant in this domain (all equivalence tests reported in Supplementary Information 1).
Since the preregistered models included three random-effects factors and their corresponding random slopes, we removed the correlations between slopes and intercepts across all models to reduce model complexity and improve model efficiency.
We had also preregistered the use of DFBeta diagnostics to assess model stability by iteratively removing each level of the grouping factor (i.e., participants) and examining the impact on model coefficients. However, this approach proved infeasible due to the computational burden and extended runtimes involved in refitting the full mixed-effects models for each of the 80 participants. Our models included multiple random effects and interaction terms, and convergence issues were frequently encountered during even a single model fit, making iterative refitting impractical and unreliable. As a practical alternative, model stability was evaluated in the explorative analyses using data averaged across trials (model stability values are reported in the results table in Supplementary Information 2).
Valence classification task
To assess whether our experimental setup successfully induced perceptual uncertainty, we examined mean accuracy across the first two blocks for each participant in the social and control cue conditions. The majority of participants (N = 63) exhibited higher accuracy in the social cue condition compared to the control cue condition, indicating their reliance on social cues to resolve perceptual uncertainty. However, a subset of participants (N = 17) performed better in the control cue condition than in the social cue condition, potentially reflecting individual differences in strategy use or a lack of trust in the social cue within a lab-based artificial social learning paradigm. These participants may have perceived the facial expressions as unreliable or irrelevant, opting instead for their own subjective valence judgments.
Additionally, participants often defaulted to a ‘neutral’ response when highly uncertain, which may have masked cue condition differences at the individual level. This response bias is reflected in the relatively high frequency of neutral classifications for both positive and negative target images – even in the social cue condition, where participants frequently chose the neutral option despite the presence of facial expressions (social: N = 6059; control: N = 9300). Despite individual variability in the valence classification task, the planned statistical model revealed significantly higher accuracy for emotional target images in the social compared to the control cue condition. This suggests that most participants incorporated the social cues into their decision-making. Based on this overall pattern, no participants were excluded from subsequent analyses based on their initial performance in the first two blocks.
Valence Classification Accuracy and RT (Fig. 2a and b): A significant valence × cue condition interaction emerged for classification accuracy. Relative to neutral target images, accuracy was significantly higher in the social compared to the control cue condition for both negative and positive target images (negative: β = 1.667, SE = 0.425, z = 3.919, p <.001; positive: β = 1.551, SE = 0.421, z = 3.684, p <.001). For response times (RTs), we also observed a valence × cue condition interaction, again relative to neutral target images; however, the effect was contrary to expectations. Rather than faster responses, the presence of a social cue was associated with marginally (p-value made conservative after Type 1 correction) longer RTs for positive target images (positive: β = 0.012, SE = 0.006, t = 2.111, p =.040).
While overall performance improved across trials, the model revealed no significant cue condition × trial interactions for either valence classification accuracy or RTs, contrary to our predictions. However, a significant valence × trial interaction emerged for accuracy for the negative target images, where accuracy gains over time were reduced compared to neutral ones (negative: β = −0.262, SE = 0.088, z = −2.972, p =.003). This pattern suggests a reduced perceptual learning effect for negative target images. For RTs, a main effect of trial was observed, with RTs decreasing across trials regardless of valence or cue condition (trial: β = −0.045, SE = 0.005, t = − 9.593, p <.001), indicating general task-related learning and improved processing efficiency over time.
Fig. 2. Day 1 behavioral results. (a) Valence Classification accuracy across trials. (b) Valence classification RTs aggregated across trials. Plots indicates mean model-based predicted values along with the corresponding 95% CIs.
To assess whether participants formed associations between the target images and their respective cue conditions, we administered recall and recognition memory tasks immediately after the learning session. Performance on both tasks exceeded chance levels (recall: 25%; recognition: 50%), indicating successful learning across participants. In the recall task (Fig. S1a – see Supplementary Information 1), mean accuracy was higher for target images in the social cue condition (67.98%) than in the control cue condition (59.27%), suggesting enhanced memory encoding when social cues were present. In contrast, recognition accuracy (Fig. S1b - see Supplementary Information 1) was comparable between the social (72.20%) and control (72.13%) cue conditions, indicating no cue condition-specific differences in recognition memory. Statistical analyses for the Day1 memory tests (recall and recognition) are reported below.
Day 1 Memory Test – Recall (**Fig. S1a –**see Supplementary Information 1): The probability of correctly recalling the facial expression paired with each target image during the learning session was significantly modulated by a valence × cue condition interaction. Although recall performance in the social cue condition was only slightly higher than in the control cue condition for neutral images, this difference was substantially larger for positive images. Specifically, positive images paired with social cues during learning were recalled more accurately than those paired with control cues (β = 0.843, SE = 0.224, z = 3.769, p <.001).
Day 1 Memory Test – Recognition (**Fig. S1b –**see Supplementary Information 1): The probability of correctly judging whether a target image was presented with the same or a different facial expression relative to learning session was also influenced by a valence × cue condition interaction. For neutral images, performance in the social cue condition was slightly lower than in the control cue condition. In contrast, for positive images, recognition accuracy was significantly higher in the social cue condition compared to the control cue condition (β = 0.572, SE = 0.262, z = 2.187, p =.029).
ERP amplitudes – valence classification task (learning session)
P1 (Fig. 3a): Due to substantial inter-individual variability in P1 onset (ranging from around 50 to 120 ms), we identified the peak amplitude within this window for each participant and trial. To obtain a stable estimate, mean amplitude was calculated by averaging across a ± 10 ms window around the individual peak. These single-trial P1 amplitudes from the relevant ROIs were entered into a LMM.
The comparison between the full model and a null model (including only control variables and random effects) was not statistically significant, suggesting that the inclusion of fixed effects did not improve model fit (χ² = 8.880, df = 4, p =.064). Contrary to our hypothesis, the main effect of cue condition – although not significant – was in the opposite direction: P1 amplitudes were numerically lower in the social cue condition compared to the control cue condition (cue condition: β = −0.118, SE = 0.065, t = − 1.804, p =.079). No significant interactions involving cue condition, valence, or trial were observed. However, a significant main effect of trial emerged, with P1 amplitudes increasing over time, irrespective of cue condition or valence (trial: β = 0.136, SE = 0.057, t = 2.389, p =.020).
EPN (Fig. 3b): Single-trial EPN amplitudes were extracted from the relevant ROIs within the 200–300 ms time window and analyzed using a LMM. Contrary to predictions, the model revealed no significant effect of cue condition on EPN amplitudes (cue condition: β = −0.086, SE = 0.061, t = − 1.409, p =.166), indicating that the presence of a social cue did not modulate early emotional attention during the learning session. Instead, EPN amplitudes were significantly modulated by a valence × trial interaction: across trials, negative target images elicited stronger EPN responses relative to the neutral target images (negative: β = −0.146, SE = 0.056, t = − 2.585, p =.012). While a similar pattern was seen for positive target images, the effect was only marginally (p-value made conservative after Type 1 correction) significant (positive: β = −0.141, SE = 0.062, t = − 2.255, p =.028).
LPC (Fig. 3c): Single-trial LPC amplitudes were extracted from the relevant ROIs within the 400–600 ms time window and analyzed using a LMM. Consistent with the EPN results, the model revealed no significant effect of cue condition (cue condition: β = 0.035, SE = 0.050, t = 0.697, p =.490), suggesting that the presence of a social cue did not modulate later-stage evaluative processing during learning. Contrarily, LPC amplitudes were significantly modulated by main effects of valence and trial. Specifically, amplitudes were overall enhanced for positive target images (positive: β = 0.555, SE = 0.191, t = 2.905, p =.005) and increased across trials, irrespective of valence or cue condition (trial: β = 0.136, SE = 0.053, t = 2.588, p =.012).
Fig. 3. Day 1 grand average waveforms across cue conditions for each of the target image valence categories (neu, neg, pos). (a) P1. (b) EPN. (c) LPC. Gray rectangular shaded region marks the time-window of the ERP. Topoplots indicate difference between cue conditions and green dots mark the corresponding ROIs for each of the ERPs. The corresponding line plots represent mean model-based predicted value across trials with shaded regions indicating 95% CIs.
Old/new classification task
Old/New Classification Accuracy and RT (Fig. S2a & S2b – see Supplementary Information 1): Old/New classification accuracy was significantly modulated by a three-way interaction between valence, cue condition, and trials. For positive target images, accuracy increased more steeply across trials in the social cue condition compared to the control cue condition, relative to the difference between cue conditions for neutral target images (positive: β = 0.745, SE = 0.301, z = 2.479, p =.013). However, interpretation of this three-way interaction is limited by ceiling effects, as recognition accuracy for positive and neutral target images was already high across cue conditions, reducing sensitivity to subtle cue condition differences. Thus, the observed interaction may reflect compressed variance rather than genuine improvements in memory retrieval over time. Old/New Classification RTs were significantly influenced by trial, with faster classification responses across repeated exposures (trial: β = −0.048, SE = 0.004, t = −12.750, p <.001). As predicted, no effect of cue condition on RTs was observed (cue condition: β = 0.000, SE = 0.001, t = −0.003, p =.999).
ERP amplitudes – Old/New Classification Task (Test Session).
P1 (Fig. 4a):Due to substantial variability in P1 onset across participants (ranging from 50 to 120 ms), we identified the peak amplitude within this window for each participant and trial. To obtain a stable estimate, mean amplitudes were calculated by averaging over a ± 10 ms window around the peak. These single-trial P1 amplitudes, extracted from the relevant ROIs, were entered into a LMM.
P1 amplitudes were significantly modulated by a main effect of trial, with amplitudes increasing over time (trial: β = 0.365, SE = 0.071, t = 5.152, p <.001). In contrast to our predictions, no significant effect of cue condition was observed (cue condition: β = 0.015, SE = 0.073, t = 0.211, p =.746).
**EPN ** (Fig. 4b): Single-trial EPN amplitudes, extracted from the relevant ROIs within the 200–300 ms time window, were analyzed using a LMM. EPN amplitudes were significantly modulated by a valence x cue condition interaction, with more pronounced amplitudes for positive target images in the social cue condition compared to the control cue condition, relative to the difference between cue conditions for neutral target images (positive: β = −0.422, SE = 0.172, t = −2.453, p =.017).
Additionally, a marginal (p-value made conservative after Type 1 correction) valence x trial interaction emerged across both cue conditions, indicating that emotional target images consistently elicited stronger EPN amplitudes across trials relative to neutral target images (negative: β = −0.148, SE = 0.072, t = −2.070, p =.044; positive: β = −0.151, SE = 0.069, t = −2.195, p =.033).
LPC (Fig. 4c): Single-trial LPC amplitudes, extracted from the relevant ROIs within the 400–600 ms time window, were analyzed using a LMM. LPC amplitudes were modulated by an interaction of valence x trials, with marginally (p-value made conservative after Type 1 correction) significant extinction effects observed for negative target images across trials, relative to the neutral target images (negative: β = −0.132, SE = 0.062, t = −2.111, p =.040). No significant effect of cue condition emerged (cue condition: β = 0.057, SE = 0.050, t = 1.126, p =.263).
Fig. 4. Day 2 grand average waveforms across cue conditions for each of the target image valence category (neu, neg, pos). (a) P1. (b) EPN. (c) LPC. Gray rectangular shaded region marks the time-window of the ERP. Topoplots indicate difference between cue conditions and green dots are the corresponding regions of interest (ROIs) for each of the ERPs. The corresponding line plots represent mean model predicted values aggregated across trials with error bars regions indicating 95% CIs.
Old/new effects:
To examine the Old/New effect—typically characterized by more positive ERP amplitudes for previously seen (“old”) images compared to unseen (“new”) images in the P300 and LPC time windows—we conducted LMM analyses. Due to missing triggers for some new images, trial-wise EEG data could not be extracted. Thus, the models were fitted using average amplitudes for each valence-by-cue condition combination. Analyses were restricted to old images in the control cue condition, which were then compared to new images.
**P300 **(Fig. S3a – see Supplementary Information 1): As expected, old images elicited significantly greater positive amplitudes than new images (old: β = 0.861, SE = 0.071, t = 12.065, p <.001). This old/new effect was more pronounced for emotional target images relative to neutral images (negative: β = 1.107, SE = 0.087, t = 12.667, p <.001; positive: β = 0.244, SE = 0.071, t = 3.425, p =.005).
LPC(Fig. S3b – see Supplementary Information 1): Similarly, LPC amplitudes were significantly larger for old than new images (old: β = 0.660, SE = 0.057, t = 11.501, p <.001), with greater enhancement for emotional target images compared to neutral images (negative: β = 0.709, SE = 0.070, t = 10.082, p <.001; positive: β = 0.287, SE = 0.070, t = 4.084, p <.001).
Exploratory results
Identical to the preregistered models, we removed the correlations between slopes and intercepts across all models to reduce model complexity and improve model efficiency.
All LMMs were fit using the “bobyqa” optimizer, and the GLMMs used “nloptwrap”.
Valence classification task – learning session
RTs accounting for classification accuracy (Fig. 5): RT analyses revealed a valence × classification accuracy interaction. Specifically, inaccurately classified positive target images – those judged as neutral—elicited significantly longer RTs relative to the difference between cue conditions observed for the neutral target images (positive: β = 0.057, SE = 0.014, t = 4.041, p <.001). A similar trend was observed for inaccurately classified negative images, but this effect did not reach statistical significance (negative: β = 0.021, SE = 0.015, t = 1.416, p =.163).
Fig. 5. Valence Classification RTs split by classification accuracy. Plots represent mean model-based predicted values with their corresponding 95% CIs as error bars.
P1 Amplitudes Accounting for Classification Accuracy (Fig. 6a): P1 amplitudes, referenced to neutral target images, were significantly modulated by a three-way interaction between valence, cue condition, and classification accuracy for negative target images (negative: β = 1.280, SE = 0.468, t = 2.737, p =.006), and showed an effect in the similar direction – however not significant - for positive target images (positive: β = 0.901, SE = 0.479, t = 1.877, p =.060). For neutral target images, P1 amplitudes were lower in the social compared to the control cue condition, regardless of classification accuracy. For negative target images, P1 amplitudes were similarly reduced in the social cue condition compared to the control cue when classifications were accurate. However, this pattern reversed for inaccurately classified negative images, where P1 amplitudes were enhanced in the social cue condition relative to the control cue condition.
EPN Amplitudes Accounting for Classification Accuracy (Fig. 6b): EPN amplitudes were significantly modulated by normative valence, with enhanced amplitudes for both negative and positive target images compared to neutral images. (negative: β = −0.778, SE = 0.146, t = −5.318, p <.001; positive: β = −0.582, SE = 0.145, t = −4.023, p <.001). No significant effects of classification accuracy were observed. However, when accounting for classification accuracy, a trend emerged toward greater EPN amplitudes in the social compared to the control cue condition (β = −0.180, SE = 0.098, t = −1.840, p =.069).
LPC Amplitudes Accounting for Classification Accuracy (Fig. 6c): LPC amplitudes were significantly modulated by a valence x classification accuracy interaction: Specifically, inaccurately classified emotional target images elicited reduced amplitudes compared to accurately classified ones, relative to the pattern observed for neutral target images (negative : β = −0.582, SE = 0.205, t = −2.838, p =.006; positive : β = −1.131, SE = 0.226, t = −4.996, p <.001).
Fig. 6. Day 1 mean ERP Amplitudes from corresponding windows and regions of interest (ROIs). *a.P1 b. EPN c.*LPC. ERPs split by classification accuracy. Plots represent mean model-based predicted values with their corresponding 95% CIs as error bars.
Old/new classification task –test session
Old/New Classification Accuracy Accounting for Classification Accuracy (Fig. S4 – see supplementary Information 1): Old/New classification accuracy was modulated by an interaction between valence x classification accuracy, relative to neutral images. Specifically, emotional images that had been inaccurately classified on Day 1 were significantly less likely to be remembered than accurately classified images (negative: β = −1.503, SE = 0.240, z = −6.276, p <.001; positive: β = −1.461, SE = 0.255, z = −5.734, p <.001).
EPN Amplitudes Accounting for Classification Accuracy (Fig. 7a): EPN amplitudes, in reference to the neutral target images, were significantly modulated by a three-way interaction between valence, cue condition, and classification accuracy for the negative target images (negative : β = −0.978, SE = 0.470, t = −2.082, p =.040). To disentangle these effects, separate models were fitted for accurately and inaccurately classified images.
For inaccurate classifications, EPN amplitudes for neutral target images were less pronounced in the social cue condition compared to the control cue condition, In contrast, EPN amplitudes for both negative and positive target images were enhanced but comparable to those in the control cue condition (negative: β = −0.692, SE = 0.340, t = −2.034, p =.045; positive: β = −0.753, SE = 0.359, t = −2.098, p =.039). For accurate classifications, EPN amplitudes for neutral images remained comparable across cue conditions. However, for negative target images, the social cue condition elicited numerically – but not significantly – smaller EPN amplitudes compared to the control cue condition (negative: β = 0.282, SE = 0.319, t = 0.882, p =.380). In contrast, accurately classified positive target images continued to elicit enhanced EPN amplitudes in the social compared to the control cue condition (positive: β = −0.638, SE = 0.268, t = −2.382, p =.018).
However, the three-way interaction involving cue condition did not survive false-discovery–rate (FDR) correction, and therefore these cue condition-related effects should be interpreted with caution.
LPC Amplitudes Accounting for Classification Accuracy (Fig. 7b): LPC amplitudes were significantly modulated by a valence x cue condition interaction for negative target images. Specifically, while amplitudes were comparable between cue conditions for neutral target images, they were reduced in the social compared to the control for negative target images (negative: β = −0.272, SE = 0.132, t = −2.056, p =.043). Consistent findings from the learning session, LPC amplitudes were also influenced by a valence x classification accuracy interaction. Relative to the pattern observed for neutral target images, inaccurate classifications of emotional target images led to reduced LPC amplitudes compared to accurate classifications (negative: β = −0.830, SE = 0.145, t = −5.723, p <.001; positive: β = −0.637, SE = 0.144, t = −4.429, p <.001).
Importantly, the valence × cue condition effect on LPC amplitudes did not survive FDR correction, whereas the effects involving classification accuracy remained robust. Therefore, the cue condition-related LPC differences on Day 2 should be considered preliminary and interpreted cautiously.
Fig. 7. Day 2 mean ERP Amplitudes from corresponding windows and regions of interest (ROIs). a. EPN b.LPC. ERPs split by classification accuracy. Plots represent mean model-based predicted values with their corresponding 95% CIs as error bars.
Discussion
The present study examined whether emotional salience can be transferred from social cues – specifically, facial expressions – to ambiguous visual stimuli under conditions of experimentally induced perceptual uncertainty. During the learning session, participants performed a valence classification task in which some images were paired with emotional facial expressions (social cue condition) while others were paired with scrambled versions of faces that carried no emotional content (control cue condition). On the following day, preservation of emotional salience in memory was tested in an Old/New classification task.
We preregistered the hypothesis that emotional salience would be transferred from facial expressions to target images during learning. Specifically, we expected cue condition-specific modulations of early (P1), mid (EPN), and late (LPC) ERP components over repeated exposures to reflect this associative process. However, these predictions were not supported. ERP amplitudes during the learning session did not differ by cue condition, and equivalence tests confirmed that the observed effects were statistically equivalent to zero within predefined bounds.
However, this pattern does not contradict the theoretical rationale motivating the study. The paradigm was explicitly designed to induce perceptual uncertainty — a state under which learners are expected to rely more heavily on social cues. The present findings suggest that this perceptual uncertainty was not uniform but instead varied substantially across trials and images, producing considerable variability in how confidently participants perceived the valence of the target stimuli.
The exploratory analyses incorporating classification accuracy as an index of subjective perceptual certainty help illuminate this heterogeneity. These analyses indicate that social cue effects on both early and late stages of stimulus processing emerge selectively under conditions of subjective metacognitive uncertainty, when participants were least confident in their own perceptual judgments.
Relative to the neutral target images, social cues were significantly used for valence classification of emotional target images. However, ERP evidence did not indicate emotional salience transfer across trials as predicted. Instead, the influence of social cues during the learning session appeared to vary depending on participants’ metacognitive state of subjective uncertainty as reflected in the P1 modulations in response to negative target images. Recognition memory after overnight consolidation showed evidence of additive emotional salience from the social cues at the EPN level for accurately classified positive target images in comparison to the neutral target images. In later stages of processing, LPC amplitudes were reduced for negative target images in the social cue condition, irrespective of classification accuracy. Together, our findings suggest that the influence of social cues is contingent on the metacognitive state of subjective uncertainty and does not automatically induce emotional salience transfer during learning. This pattern became emergent in the test session after overnight consolidation: Positive target images – associated with lower subjective uncertainty – showed enhanced emotional salience, while negative target images, marked by higher uncertainty, led to greater reliance on social cues and weaker encoding of the image itself.
Day 1: learning session - valence classification task
As predicted, social cues significantly improved valence classification accuracy, particularly for emotional target images. This finding aligns with previous findings suggesting that social information becomes especially valuable under cue conditions of uncertainty, ambiguity, or when individuals seek external guidance for decision-making^4,59–61,67^. However, contrary to our expectations, these accuracy benefits did not increase over trials. Instead, both classification accuracy and RTs improved similarly across social and control cue conditions – suggesting a general effect of perceptual learning. Previous studies have demonstrated trial-wise calibration of advice or social inputs leading to cumulative learning^30,85^. In contrast, our paradigm offered limited incentives for such calibration: target images were repeatedly paired with the same social cue, and participants did not receive feedback about their classification accuracy. Moreover, the inclusion of a ‘neutral’ response option allowed participants to indicate subjective uncertainty explicitly – even when a social cue was available—potentially diluting any consistent reliance on external cues over time.
A general trend of slower RTs in the social cue condition compared to the control cue condition for positive target images support the interpretation that social cues were used selectively based on participants’ metacognitive state. This result deviates from our preregistered prediction that social cues would reduce uncertainty and thereby facilitate faster responses. Instead, the prolonged RTs likely reflect additional cognitive demands, with participants engaging in more deliberate evaluative processes – potentially weighing their subjective uncertainty against the valence information provided by the social cue. Consistent with this interpretation, exploratory analyses accounting for classification accuracy as a proxy for subjective uncertainty showed slower responses for inaccurate compared to accurate classifications, particularly for positive target images, suggesting that accurate classifications were associated with greater certainty than inaccurate ones.
Neural modulation during early perceptual processing, as indexed by the P1 ERP component, did not support the predicted transfer of emotional salience from social cues to the ambiguous target images. Previous studies indicated that social cues can modulate early attentional processes, particularly under conditions of uncertainty or social pressure^66,68–71^. In line with this, our data revealed metacognitive state-dependent modulations of the P1 component.
The preregistered models for P1 indicated a pattern for reduced amplitudes in the social compared to the control cue condition, however accounting for classification accuracy as a proxy for subjective uncertainty helped tease apart the P1 modulations better. For neutral target images, P1 amplitudes were reduced across both social and control cue conditions when classifications were accurate, consistent with low subjective uncertainty and reduced perceptual demands for neutral target images. When participants misclassified neutral images as emotional, P1 responses diverged: amplitudes increased in the control cue condition—indicating heightened sensory engagement under uncertainty—but decreased in the social cue condition, suggesting that participants used the facial expression to resolve uncertainty and suppress early visual processing.
A complementary pattern emerged for emotional target images, particularly negative ones. Behavioral findings showed weaker perceptual learning across trials for negative compared to neutral target images, suggesting less stable internal judgments and a greater reliance on social cues. This was mirrored in P1 amplitudes: when negative images were accurately classified in the social cue condition, amplitudes were reduced compared to control cue condition, suggesting facilitated classification through social cue alignment. Conversely for inaccurately classified emotional images, P1 amplitudes were increased in the social cue condition, indicating greater sensory engagement under uncertainty – internal and external signals conflicted.
These findings are consistent with predictive coding models^86,87^, which propose that prior information – including social cues – modulates early sensory processing based on the magnitude of the prediction error. When internal expectations and external cues align, prediction error is low and sensory responses are suppressed; when they conflict, prediction error is high, prompting greater sensory engagement. Thus, rather than directly shaping early perceptual representations, social cues appear to exert a flexible, metacognitive influence – either dampening or enhancing early visual responses depending on subjective uncertainty^4,60,61,67^.
No evidence was found for the predicted transfer of emotional salience from social cues to target images at mid or late stages of processing, as indexed by the EPN and LPC components. Instead, both ERP markers were modulated by emotional valence regardless of cue condition.
The EPN amplitudes in both the preregistered and the exploratory models showed significantly greater amplitudes for emotional compared to neutral target images, consistent with its established role in bottom-up attentional prioritization of emotionally salient stimuli across different stimulus domains^46–51^. The absence of any cue condition effect suggests that emotional salience may have been driven primarily by intrinsic stimulus properties, with limited added value from the social cues. This aligns with previous findings that EPN modulations are by low-level perceptual features and are often insensitive to context or task demands^48,88,89^. Although we controlled for image complexity in our analyses, it remains possible that residual perceptual differences contributed to the observed valence effects—particularly relevant given the experimentally induced perceptual uncertainty in our task.
The LPC component revealed robust effects of classification accuracy: amplitudes were consistently higher for correctly classified emotional images than for misclassified ones. This finding supports the interpretation of the LPC as a marker of sustained evaluative processing and affective memory encoding^46,48,49,52,53,53–55^. Notably, LPC amplitudes were unaffected by social cue condition, suggesting that late-stage emotional evaluation was driven primarily by internal decision certainty rather than external cue influence.
Previous studies have shown that inherently neutral stimuli—such as pseudowords or neutral faces—can acquire motivational salience through associations with gain or losses, as reflected in EPN and LPC modulations^40–43,90,91^. In those paradigms, participants lacked prior affective evaluations of the stimuli, which likely facilitated stronger and more consistent associative effects. In contrast, the target images in our study allowed for subjective valence judgments, which may have influenced how social cues were interpreted and used. Rather than forming new associations, participants may have used facial expressions as secondary inputs in the appraisal process—adjusting or affirming their initial judgments depending on their level of subjective uncertainty. Future research could test this possibility more directly by restricting stimulus content to truly neutral items and assessing whether emotional social cues can more reliably induce associative salience under such conditions.
Day 2: test session – old/new classification task
Recognition accuracy for emotional target images was significantly modulated by the classification accuracy during the learning session. Specifically, target images that were correctly classified during the learning session on Day 1 were more likely to be recognized in the test session on Day 2. This suggests that successful encoding of emotional content – whether based on internal judgments or supported by cues –facilitated subsequent memory, consistent with well-established links between affective salience and memory^92–95^. Importantly, no additional memory benefit emerged for target images paired with social cues. This aligns with the ERP results from the learning session, where LPC amplitudes were driven by classification accuracy rather than cue condition, indicating that memory formation was more depended on the strength of subjective emotional evaluation than on the presence of social cues.
In contrast to the P1 modulations observed during the learning session, no significant differences in P1 amplitudes were found during the test session – neither as a function of cue condition nor valence. Although the P1 component is often associated with early visual attention, its sensitivity to emotional content has been inconsistent across studies. While some have reported enhanced P1 responses for emotional stimuli (e.g^96,97^., others have failed to find reliable effects of emotion on the P1 (e.g^58,90,98^. Our findings align with the latter, suggesting that neither emotional valence nor prior social association influenced early perceptual processing.
This result diverges from previous associative learning studies, where P1 modulations were observed for stimuli paired with motivational outcomes – even when those incentives were no longer present during subsequent old/new recognition tasks^40–43,99^. However, other studies have also failed to observe early ERP effects of learned salience under similar test conditions, particularly when associations were formed in cross-modal paradigms or involved socially or affectively complex cues^98,100^.
In contrast to the absence of cue condition effects on the EPN during the learning session, the preregistered model for the Day 2 test session revealed enhanced EPN amplitudes for the positive images in the social cue condition, relative to the control. Accounting for classification accurately, the effect was primarily driven by the accurately classified positive target images. No such effect was found for negative target images. This cue condition-specific effect, measured relative to the difference observed between cue conditions for neutral images, suggests that positive images previously paired with social cues exerted a lasting influence on early emotional attention, even when the social cue was no longer present and the task did not require emotional classification. However, the effect of classification accuracy on EPN amplitudes did not survive the FDR correction and should therefore be interpreted with caution.
These findings are consistent with Schacht & Vrtička, 2018^101^ who showed that positive emotional images depicting social content elicited enhanced EPN amplitudes, effectively counteracting the typical mid-stage negativity bias. In their study, a significant interaction between emotional valence and social content of the stimuli revealed that positive social stimuli elicited stronger attentional engagement, whereas no such enhancement was found for negative images^102^.
Further support for a temporal dissociation between positive and negative motivational outcomes comes from Grassi et al., 2024^103^, who found that loss-associated stimuli influenced ERP responses during the initial learning phase, while gain-associated stimuli modulated EPN amplitudes only during a later test phase. These findings suggest that losses primarily shape early encoding processes, while gains exert stronger effects at retrieval^42^. A similar temporal dynamic is evident in our results: negative social cues appeared to influence early perceptual classification (reflected in P1 modulations), whereas only positive social cues enhanced emotional processing after memory consolidation, as indexed by elevated EPN amplitudes during Day 2. This temporal dissociation highlights the distinct neural trajectories by which positive and negative social information contribute to affective learning and memory.
Positive social cues may have been particularly effective because of their inherently rewarding nature. Prior studies suggest that happy facial expressions function as social reinforcers, akin to primary rewards like money^90,104,105^, and are often interpreted as broadly rewarding rather than as specific valence signals. This interpretation is supported by activation of reward-related brain areas, such as the nucleus accumbens and the medial prefrontal cortex, in response to positive stimuli^106^. The observed EPN enhancement for accurately classified positive images in the social cue condition may reflect a broadened attentional state during encoding – consistent with the broaden-and-build theory of positive emotion^107,108^. By expanding attentional scope and supporting richer associative processing, positive social cues likely enhanced consolidation and promoted prioritized reactivation during later retrieval^109^.
In contrast to the additive salience effects for positive target images at the EPN level, negative target images elicited reduced LPC amplitudes in the social compared to the control cue condition. Although this effect did not survive FDR correction and should therefore be interpreted with caution, its pattern is theoretically informative. This suggests a different encoding dynamic: participants may have relied more heavily on social cues when classifying negative images, likely due to weaker or less accessible internal valence representations. Early P1 modulations support this interpretation, pointing to social cue-driven perceptual facilitation. However, such reliance on external cues may have come at the cost of deeper stimulus processing. From a metacognitive perspective, heightened uncertainty may have shifted attention away from the image itself, resulting in attenuated elaborative encoding and weaker memory traces—consistent with prior work showing that negative affect can amplify early processing while dampening later task-related stages^110^. In contrast, for positive target images, stronger internal signals that aligned with the social cue likely supported deeper encoding and greater emotional salience at retrieval.
While previous research on social appraisal has demonstrated that emotional evaluations of stimuli can be shaped by concurrent social cues—often through expectation-driven or perspective-taking mechanisms^20,21,33,34^ these effects have typically been studied in single-trial designs and in the presence of a social agent. In contrast, our study examined whether emotional salience could be transferred to stimuli through repeated co-occurrence with emotional expressions and persist in the absence of the social cue. Although we found no strong evidence for such emotional salience transfer, our findings suggest that social information is flexibly incorporated into valence judgments based on the subjective uncertainty. This suggests that the influence of social cues in affective learning is metacognitively regulated, rather than automatically applied.
Several limitations should be acknowledged. First, our design relied on emotionally ambiguous stimuli, which may have introduced individual variability in how social cues were used or interpreted. This, in turn, prompted exploratory analyses that used classification accuracy as a proxy for subjective uncertainty. Further, the experimental design complicates comparisons with prior studies using inherently neutral stimuli. The use of emotional facial expressions as social cues—while ecologically valid—may have limited associative strength compared to more explicit reward or punishment outcomes. Further, although the role of metacognitive uncertainty was already outlined in the theoretical rationale of the study, our explicit incorporation of subjective classification accuracy into the statistical models occurred only in exploratory analyses. Thus, interpretations involving individual differences in subjective uncertainty should be considered preliminary and require replication. Additionally, ERP components like the EPN and LPC are influenced by both stimulus-driven and task-related factors, and disentangling these influences remains challenging in complex associative paradigms.
Lastly, the facial expressions used in the present study were static, and the task lacked several features that characterize natural social interactions. Therefore, it is possible that participants did not experience the facial expressions as genuine social reactions in the same extent as they would during live interpersonal encounters. Future studies should enhance ecological validity by incorporating dynamic expressions, interactive or contingent social cues, virtual agents, or more immersive environments to determine whether the same mechanisms of cue integration and uncertainty monitoring generalize to more naturalistic contexts. Such extensions would complement the current findings and offer a richer understanding of how social information shapes perception and learning in real-world settings.
Taken together, our findings provide novel evidence that the influence of social cues on affective learning is shaped by metacognitive factors, particularly subjective uncertainty. Rather than transferring emotional salience automatically, social cues appear to act as flexible informational resources—modulating perceptual and memory processes depending on the availability and strength of internal evidence. These results extend current theories of social appraisal and emotional learning by highlighting the importance of individual-level uncertainty in guiding the integration of social information over time.
Supplementary Information
Below is the link to the electronic supplementary material.
Supplementary Material 1
Supplementary Material 2
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Gariépy, J. F. et al. Social learning in humans and other animals. Frontiers in Neuroscience, 8. https://www.frontiersin.org/articles/ (2014). 10.3389/fnins.2014.0005810.3389/fnins.2014.00058 PMC 398206124765063 · doi ↗ · pubmed ↗
- 2Bayer, M. & Schacht, A. Event-related brain responses to emotional words, pictures, and faces – a cross-domain comparison. Frontiers in Psychology, 5. https://www.frontiersin.org/articles/ (2014). 10.3389/fpsyg.2014.0110610.3389/fpsyg.2014.01106 PMC 418627125339927 · doi ↗ · pubmed ↗
- 3Fischer, A. Learning from others’ emotions (pp. 165–184). (2019). 10.1017/9781108661362.008
- 4Dukes, D. & Clément, F. Foundations of Affective Social Learning: Conceptualizing the Social Transmission of Value (Cambridge University Press, 2019).
- 5Schupp, H. T., Flaisch, T., Stockburger, J. & Junghöfer, M. Emotion and attention: Event-related brain potential studies. In S. Anders, G. Ende, M. Junghofer, J. Kissler, & D. Wildgruber (Eds.), Progress in Brain Research (Vol. 156, pp. 31–51). Elsevier. (2006). 10.1016/S 0079-6123(06)56002-910.1016/S 0079-6123(06)56002-917015073 · doi ↗ · pubmed ↗
- 6Rolls, E. T. Emotion and Decision Making Explained. OUP Oxford. (2014).10.1016/j.cortex.2014.01.02024698794 · doi ↗ · pubmed ↗