Concurrent representations of reinstated and transformed memories and their modulation by reward

TL;DR

This study shows how the brain simultaneously reactivates original and transformed memories during retrieval, with reward enhancing the transformation process.

Contribution

The paper reveals that reward modulates the transformation of memory representations in the anterior hippocampus during retrieval.

Findings

The parahippocampal cortex reinstates scene-specific encoding patterns during successful retrieval.

The medial prefrontal cortex and anterior hippocampus activate transformed, generalized memory representations.

High reward enhances memory transformation in the anterior hippocampus.

Abstract

An integral part of episodic retrieval is the reinstatement of neural activity that was present in the medial temporal lobe during encoding. However, neural memory representations do not remain static. Consolidation promotes the transformation of representations that are specific to individual episodes toward more generalized representations that reflect commonalities across episodes. Moreover, reward has been shown to augment episodic memory by enhancing consolidation, and it may accelerate the transformation of neural memory representations. We investigated this account with n = 40 human participants using fMRI and an associative memory task. They encoded pictures of objects, each with one of four recurring scenes. Two scenes led to high reward, and two led to low reward. The next day, participants encountered the objects again and retrieved the scenes from memory. Using representational similarity analysis, we demonstrate that retrieval is concurrently accompanied by the reinstatement of original neural representations and the activation of transformed, more generalized memories. Specifically, the parahippocampal cortex reinstates scene-specific patterns from the encoding phase during successful retrieval. In contrast, activity patterns in the medial prefrontal cortex and anterior hippocampus reflect transformed memories: They become more similar to each other for memories sharing the same scene, independent of memory success. Importantly, high reward enhances memory transformation in the anterior hippocampus. The brain thus maintains complementary memory representations: An episodic representation that resembles the original encoding pattern, and a generalized representation that summarizes commonalities across memories—in part for particularly valuable information.

Figures

(A) Example trial for the incidental encoding task. After a variable inter-trial interval of 1.75–5.75 s, participants viewed one of 160 objects on top of one of four scenes, and mentally engaged in a scene-specific simulation task (here: “Would the object float on the water?”) for 3.75 s. They had 1.5 s to respond “yes” or “no”, and received reward feedback (high: 2.00 points, low: 0.02 points) for 1 s for each correct response. (B) Example trial for the scene recall task. After a variable ITI, participants viewed one of the objects from the encoding phase for 3 s and tried to retrieve the scene from memory. They then had 2 s to respond by choosing a label presented on the screen (e.g., “swimming pool”). If they chose a scene label, they then had 2 s to rate the confidence of their choice on a scale of one (“not at all confident”) to four (“very confident”). (C) Representational similarity analysis. We used two measures to assess similarity between trials that either shared a scene (same-scene similarity) or did not (different-scene similarity). To test reinstatement of scene-specific activity from the encoding phase, we assessed encoding-retrieval similarity. That is, we correlated pairs of retrieval trials with encoding trials that shared the same scene or not. To test for memory generalization based on overlapping features (i.e., the same scene), we assessed retrieval-retrieval similarity. That is, we correlated pairs of trials from the retrieval phase that had either shared a scene (same-scene similarity) or had not (different-scene similarity). We note that the depicted similarity matrices are for illustrative purposes and thus are not based on empirical data. For copyright reasons, we here display photographs that are similar to the actual experimental stimuli (scene images fromhttp://pixabay.com; object images by the investigators).

Behavioral results. Left panel: Accuracy during the incidental encoding task. Middle panel: Median response times (RTs) for correct responses during the incidental encoding task. Right panel: Accuracy during the scene recall task. Error bars indicate 95% confidence interval. n.s. indicates not significant, ~ indicates 0.05 <p< 0.1.

Encoding-retrieval similarity. Average encoding-retrieval similarity values in the three a-priori ROIs: mPFC, PHC, and HC. Notes refer to effects from three-way repeated-measures ANOVAs with the factors reward, memory, and scene overlap (see main text for details). Error bars indicate 95% confidence interval. HR: high-reward/remembered; HF: high-reward/forgotten; LR: low-reward/remembered, LF: low-reward/forgotten; same: same-scene similarity, diff: different-scene similarity;R−same¯,R−diff¯: remembered/same-scene similarity and remembered/different-scene similarity averaged across the reward factor,rrefers to mean Fisher-z transformed Pearson correlation coefficients. The zoomed-in panel clarifies the direction of the interaction effect. * indicatesp< 0.05.

Retrieval-retrieval similarity. (A) Average retrieval-retrieval similarity values in the three a-priori ROIs: mPFC, PHC, and HC. (B) Average retrieval-retrieval similarity in the anterior versus posterior portion of the HC. (C) Results of whole-brain searchlight analysis for the main effect of scene overlap (same-scene similarity > different-scene similarity across levels of reward and memory). Average retrieval-retrieval similarity was extracted for visualization. Display thresholdp< 0.001 unc.,k= 5 voxels. Notes refer to effects from three-way repeated-measures ANOVAs with the factors reward, memory, and scene overlap (see main text for details). Error bars indicate 95% confidence interval. HR: high-reward/remembered; HF: high-reward/forgotten; LR: low-reward/remembered, LF: low-reward/forgotten; same: same-scene similarity, diff: different-scene similarity;high−same¯,high−diff¯: high reward/same-scene similarity and high reward/different-scene similarity averaged across the memory factor,rrefers to mean Fisher-z transformed Pearson correlation coefficients. The zoomed-in panel clarifies the direction of the interaction effect. * indicatesp< 0.05, ~ indicates 0.05 <p< 0.1.

Full text

Introduction

1

The human ability to retain memories is remarkable: Seemingly effortlessly, we can recall the events of yesterday in rich detail. Some events—for example, a picnic—we may be able to recall more easily than others – for example, a ride on the train. At the same time, we also know how picnics and train rides generally work, allowing us to easily plan for an upcoming event that may take place on the next weekend. This is because, over time, the commonalities across similar episodes are extracted and the memories are thus transformed into more generalized knowledge. Such knowledge can take the form of, for example, mental schemas, scripts, or categories (Ghosh & Gilboa, 2014;Gilboa & Marlatte, 2017). But how do these recalled and transformed memories relate to each other? And, given that some events are retained more easily than others, are some memories more likely to be transformed?

Neuroimaging has given insight into the neural representations underlying these distinct forms of memories. On the one hand, episodic recall is accompanied by reinstatement of the original encoding activity. This has especially been shown in content-sensitive regions of the medial temporal lobe (MTL), including the parahippocampal cortex (PHC) (H. Schultz, Sommer, et al., 2022;H. Schultz et al., 2019;Staresina et al., 2012) and the (posterior) hippocampus (HC) (Bone & Buchsbaum, 2021).

On the other hand, neural patterns associated with generalized memories do not reflect the encoding activity of any individual episode. Over time, memories that share common features undergo a transformation so that their neural representations become more similar. This has been shown in the medial prefrontal cortex (mPFC) and HC (Audrain & McAndrews, 2022;Tompary & Davachi, 2017).

The mPFC may thus represent transformed memories in the form of generalized knowledge structures (Ghosh & Gilboa, 2014;Gilboa & Marlatte, 2017;Milivojevic et al., 2015;Morrissey et al., 2017;Paulus et al., 2021), whereas the hippocampus may contain both types of memory representations. Notably, there is some evidence for a functional specialization within the hippocampus—with more general memories, such as an episode’s gist, being supported by the anterior HC, and more detailed episodic memories being more reliant on the posterior HC (Collin et al., 2015;Gilboa & Moscovitch, 2021;Guo & Yang, 2020;Poppenk et al., 2013;Sekeres et al., 2018; but seeTompary & Davachi, 2017).

Tompary and Davachi (2017)recently investigated memory transformation through multivariate pattern analysis of fMRI data. They had participants encode pairs of unique objects with one of four recurring scenes. Either immediately following encoding or 1 week later, participants were cued with the objects to retrieve the scenes from memory. In the mPFC as well as anterior and posterior HC, neural patterns during retrieval were more similar to each other for objects that had shared the same scene (retrieval-retrieval similarity), compared to objects that had been paired with different scenes. Critically, this was only the case during the delayed memory test, indicating that the memories underwent a transformation over time. Interestingly, these observations were driven by an increase in same-scene similarity in the mPFC and posterior HC, but a decrease in different-scene similarity in the anterior HC. The authors concluded that consolidation promotes representational convergence of memories that share overlapping features, which may be an important building block for memory generalization. We note, however, that their findings are somewhat at odds with the general/episodic distinction between anterior/posterior HC outlined in the previous paragraph.

Regarding our first overarching question—how do these distinct forms of memories, veridical and transformed, relate to each other?—several issues have remained open in the literature.

First, the reported greater retrieval-retrieval similarity for objects sharing the same scene (Tompary & Davachi, 2017) may not necessarily reflect memory generalization. Instead, it may be a byproduct of scene reinstatement, that is, the reactivation of the same scene-specific encoding pattern during retrieval (Mack & Preston, 2016;Wing et al., 2015): If the same scene encoding pattern is reinstated in two retrieval trials, these trials may then be more similar to each other, thus potentially driving retrieval-retrieval similarity. Such an effect could even increase over time, given the mPFC’s time-dependent role in memory retrieval (Barry et al., 2018;Bonnici & Maguire, 2018;Bonnici et al., 2012;Sekeres et al., 2018;Sommer, 2017). Similarly, detailed memory representations in the posterior HC (Gilboa & Moscovitch, 2021;Poppenk et al., 2013;Sekeres et al., 2018) could contribute to previously reported findings of retrieval-retrieval similarity in this region (Audrain & McAndrews, 2022;Tompary & Davachi, 2017). Here, we address this issue by concurrently examining scene reinstatement as a potential alternative account for the effect reflected in retrieval-retrieval similarity. Importantly, this approach also elucidates whether the same memory cues can elicit concurrent veridical and transformed traces of the same memory, and whether one form depends on the other (Gilboa & Moscovitch, 2021;Sekeres et al., 2018).

Second, we investigate whether generalized memory representations can be expressed even in the absence of successful retrieval of individual episodes. Given that generalization entails loss of episodic detail (Sekeres et al., 2018), we suggest that such generalized representations may be activated by an episodic retrieval cue, even if these representations do not provide sufficient detail to drive successful episodic retrieval.

Third, what is the earliest time scale that both types of memory representations could be available concurrently? Memory transformation has been demonstrated after a delay of 3 days (Audrain & McAndrews, 2022) or 1 week (Tompary & Davachi, 2017), but not immediately after encoding (Audrain & McAndrews, 2022;Tompary & Davachi, 2017). It is unclear, however, when these representations start to emerge. Sleep after encoding is accompanied by neural reactivation of the encoded material (Deuker et al., 2013;Schreiner et al., 2021;Sterpenich et al., 2021), which may be one of the earliest mechanisms of memory generalization (Kumaran et al., 2016). We, therefore, suggest that memory transformation may already be observed after 1 day, that is, one night’s sleep. At this timepoint, reinstated memories are likewise available (H. Schultz, Sommer, et al., 2022).

But are some memories more likely to be transformed than others? This is our second overarching question. Previous work has been agnostic with regards to the drivers of memory generalization. We here suggest that memory transformation is promoted by reward. As described above, neural replay during consolidation appears to be critical to generalization (Kumaran et al., 2016;Liu et al., 2019), and rewarded memoranda are preferably replayed post-encoding (Gruber et al., 2016;Sterpenich et al., 2021). In general, we thus suggest that reward may facilitate memory generalization.

Specifically, the mPFC has been implicated not only in representing generalized knowledge structures (Gilboa & Marlatte, 2017) but also in reward processing (Haber & Knutson, 2010). Indeed there is evidence that mPFC representations may be shaped by value (Baram et al., 2021;Moneta et al., 2023;Paulus et al., 2021). Similarly, the anterior HC may not only be particularly involved in representations of broad, general memories, but may also process motivationally relevant aspects of a memory—such as reward (Poppenk et al., 2013). We, thus, hypothesize that reward facilitates the representational convergence of memories that share overlapping features.

To address these questions, we conducted a 2-day fMRI study with n = 40 participants. On day 1, participants engaged in an incidental encoding task (adapted fromGruber et al., 2016), in which they associated a series of single objects with one of four recurring scenes. Two of the scenes led to high reward, and the two other scenes to low reward. This task was chosen because it previously yielded a robust effect of reward on memory (Gruber et al., 2016). The next day, participants returned for a surprise scene recall task (adapted fromTompary & Davachi, 2017). Here, participants were cued with each object to recall the associated scene.

Within regions of interest (ROIs: mPFC, PHC, [anterior and posterior] HC), we tested for the presence of reinstated as well as transformed memories. We expected scene reinstatement in the PHC and HC (specifically the posterior portion), but not the mPFC: Successful retrieval should be associated with greater encoding-retrieval similarity for trials that share the same scenes, compared to those that share different scenes. Concurrently, we expected that the mPFC and HC, but not the PHC, represent transformed memories already after a 1-day delay. This would be reflected in greater retrieval-retrieval similarity for objects that had shared the same encoding scene. In the case of the HC, we would expect such an effect predominantly in the anterior part. This follows from recent proposals for functional differences along the hippocampal long axis (Gilboa & Moscovitch, 2021;Poppenk et al., 2013;Sekeres et al., 2018). However, considering the findings byTompary and Davachi (2017), such an effect may also emerge in the posterior HC. We will further test whether these effects could be driven by scene reinstatement, and how they relate to overt memory success. Lastly, we expected effects of memory transformation in the mPFC and anterior HC to be greater for highly rewarded memories.

Method

2

Sample

2.1

A total of n = 42 volunteers took part in the study. Of these, two were excluded from data analysis (one due to equipment failure, one did not return for the second day). We thus report data from n = 40 participants (mean age: 26.25 years, age range: 19–35 years, 28 women, 12 men). They were native German speakers with normal or corrected-to normal vision and without a history of psychiatric or neurological disorder. The study protocol was approved by the ethics committee of the medical faculty of the University of Leipzig (171/19-ek), and all participants provided written informed consent prior to participating. They received 9€/h and an additional bonus of up to 15€, depending on their performance during the encoding task.

Procedure and tasks

2.2

Participants took part in two fMRI sessions on consecutive days (mean delay between fMRI sessions: 22 h 35 min, range: 19 h 30 min–26 h 0 min). On day 1, they engaged in the incidental encoding task; on day 2, they returned for the surprise scene recall task.

Day 1: Incidental encoding task

2.2.1

The incidental encoding task (Fig. 1A) was adapted fromGruber et al. (2016)(see alsoSchultz et al., 2023). In each trial, participants viewed one of 160 objects paired with one of four recurring scenes: A circus, a basketball court, an office, and a swimming pool. Participants were asked to mentally simulate a scene-specific action and respond to a corresponding question (e.g., “Would the object float on water?” for the swimming pool scene). Correct responses yielded a high or low reward. Importantly, two of the scenes were always paired with high reward (2 points), while the other two scenes were always paired with low reward (0.02 points). Points were later converted to monetary reward. Allocation of scenes to reward magnitudes were instructed prior to scanning, and they were counterbalanced across participants. Each object-scene pair was presented twice, rather than once (Gruber et al., 2016). This was to increase next-day retention and foster generalization (Tompary & Davachi, 2017). The repetition occurred within the same run. This yielded a total of 320 trials over four runs. Allocation of the 160 objects to the four scenes was randomized for each participant. The order of the object-scene pairs was pseudo-randomized such that 1) each scene occurred equally often in each run, and 2) all object-scene pairs in a run were presented once before any repetitions. Diverging fromGruber et al. (2016), there was no blocking of scenes or reward magnitudes. This was to increase variance of value and/or prediction errors that potentially drive memory enhancement (e.g.,Rouhani & Niv, 2021).

(A) Example trial for the incidental encoding task. After a variable inter-trial interval of 1.75–5.75 s, participants viewed one of 160 objects on top of one of four scenes, and mentally engaged in a scene-specific simulation task (here: “Would the object float on the water?”) for 3.75 s. They had 1.5 s to respond “yes” or “no”, and received reward feedback (high: 2.00 points, low: 0.02 points) for 1 s for each correct response. (B) Example trial for the scene recall task. After a variable ITI, participants viewed one of the objects from the encoding phase for 3 s and tried to retrieve the scene from memory. They then had 2 s to respond by choosing a label presented on the screen (e.g., “swimming pool”). If they chose a scene label, they then had 2 s to rate the confidence of their choice on a scale of one (“not at all confident”) to four (“very confident”). (C) Representational similarity analysis. We used two measures to assess similarity between trials that either shared a scene (same-scene similarity) or did not (different-scene similarity). To test reinstatement of scene-specific activity from the encoding phase, we assessed encoding-retrieval similarity. That is, we correlated pairs of retrieval trials with encoding trials that shared the same scene or not. To test for memory generalization based on overlapping features (i.e., the same scene), we assessed retrieval-retrieval similarity. That is, we correlated pairs of trials from the retrieval phase that had either shared a scene (same-scene similarity) or had not (different-scene similarity). We note that the depicted similarity matrices are for illustrative purposes and thus are not based on empirical data. For copyright reasons, we here display photographs that are similar to the actual experimental stimuli (scene images fromhttp://pixabay.com; object images by the investigators).

Day 2: Scene recall task

2.2.2

The scene recall task (Fig. 1B) was adapted fromTompary and Davachi (2017)(see alsoSchultz et al., 2023). In each trial, one of the objects from the encoding task was presented, and participants were asked to recall the scene that it had been paired with. They responded with one of five choices (verbal labels for the four scenes plus a “don’t know” option). If they responded with a scene, they were also asked to rate the confidence of their choice on a scale from one (not at all) to four (very). Each object was presented once, yielding a total of 160 trials across four runs.

Day 2 also included an unscanned recall task for the reward magnitude (Schultz et al., 2023), as well as questionnaires. These are not part of the present report.

Implementation

2.2.3

Tasks were implemented in Octave (RRID:SCR_014398) and the Psychophysics Toolbox (RRID:SCR_002881). We note that, due to a bug in the code, object images in both the incidental encoding task and scene recall task were stretched to an approximately 4:3 format.

Behavioral analysis

2.3

For the incidental encoding task, we computed proportions of correct responses as well as median response times for correct responses, separately for high and low reward trials. For the scene recall task, we computed proportions of correct scene recall, separately for high and low reward trials. All further analyses included only trials for which the encoding task had been answered correctly on both repetitions. We tested for effects of reward using pairedt-tests on all three behavioral measures. Statistical analyses were conducted in R (RRID:SCR_001905) and RStudio (RRID:SCR_000432).

MRI acquisition

2.4

MRI data were acquired on a Siemens Prisma 3T system. Functional data were scanned using a whole-brain T2*-weighted gradient-echo, echo-planar pulse sequence (2 mm isotropic voxels, 72 interleaved slices, TR = 2,000 ms, TE = 25 ms, multiband acceleration factor = 3). On each day, five functional runs were acquired: One run of 120 volumes of rest followed by either four runs of 526 volumes (task plus rest, day 1) or 206 volumes (task, day 2). Gradient-echo fieldmaps were acquired at the beginning of each session. On day 1, we also acquired a T1-weighted structural image (MPRAGE, 1 mm isotropic voxels). The resting-state data as well as additional structural scans (DWI, MP2RAGE) were not analyzed for the present report.

MRI preprocessing and first-level statistics

2.5

The MRI data were first converted to the Brain Imaging Data Structure (BIDS) (Gorgolewski et al., 2016). Preprocessing was performed using fMRIPrep 21.0.2 (Esteban, Markiewicz, Goncalves, et al., 2022;Esteban et al., 2019) (RRID:SCR_016216) based on Nipype 1.6.1 (Esteban, Markiewicz, Burns, et al., 2022;Gorgolewski et al., 2011) (RRID:SCR_002502).

The T1-weighted image (T1w) was corrected for intensity non-uniformity, skull-stripped, and segmented into gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF). It was then normalized to standard space (MNI152NLin2009cAsym). From the functional data, first, a reference image was estimated for use in the motion correction and co-registration steps. The functional data were slice-time corrected to the middle temporal slice, motion-corrected, and corrected for susceptibility distortions using the fieldmap acquired at the start of each session. Functional data were then co-registered to the T1w using boundary-based registration (Greve & Fischl, 2009) with six degrees of freedom (for further preprocessing details, seehttps://fmriprep.org/en/21.0.2/).

We conducted the further processing in MATLAB (RRID:SCR_001622) and SPM12 (RRID:SCR_007037). Specifically, we set up two sets of first-level general linear models (GLMs) on the unsmoothed, non-normalized data: one set for the encoding and one for the retrieval data. Each trial was estimated in a separate GLM (Mumford et al., 2012), with a single regressor on the simulation onset and recall onset, respectively. Each model also contained categorical regressors for all other onsets, separately for each of the conditions (HR: high reward/remembered, HF: high reward/forgotten, LR: low reward/remembered, LF: low-reward/forgotten) as well as a categorical regressor encompassing all button presses. All of these regressors were convolved with the hemodynamic response function (HRF). Additionally, each model included a set of seven non-convolved noise regressors extracted during preprocessing, that is, the six rigid motion regressors (three translations, three rotations) as well as framewise displacement. Functional runs were concatenated, and session constants were included in the models. The resulting beta maps for each trial of the encoding and retrieval sessions were converted totmaps. Finally, thetmaps were minimally smoothed with a Gaussian kernel of 2 mm full width at half maximum (Dimsdale-Zucker & Ranganath, 2018).

Regions of interest

2.6

We employed bilateral anatomical masks of the PHC, the whole HC as well as its anterior and posterior subdivisions, and mPFC. For the PHC, anterior HC, and posterior HC masks, we automatically segmented each participant’s T1w using ASHS (Yushkevich et al., 2015) and the Penn Memory Center 3T ASHS Atlas for T1-weighted MRI (Xie et al., 2016). We reviewed each individual segmentation and found that the automated process led to a successful outcome for our participant population.

The PHC masks were then manually adjusted according to guidelines suggested byPruessner et al., (2002). The anterior and posterior HC masks were also combined into a single HC mask. All masks were then resampled to each participant’s functional space. For visualization (Figs. 3and4A, B), the masks were warped into standard space using the transformation matrix from the T1w normalization, and averaged across participants.

For the mPFC mask, we used the Brainnetome atlas (Fan et al., 2016) and combined left and right medial area 11, 13, and 14 as well as left and right subgenual area 32. This yielded a rather broad mask, because reward and schema effects have been observed fairly distributed across the medial parts of the ventral and rostral prefrontal cortex (Gilboa & Marlatte, 2017;Jauhar et al., 2021). The mPFC mask was warped from standard space into each participant’s single subject space using the inverse transformation from the T1w normalization and the functional reference image (see above). We note thatTompary and Davachi (2017)used a functional mask of the mPFC based on a successful memory contrast. We chose an anatomical mask instead, because we also tested the role of memory success and thus avoided biasing our analysis in that direction.

Representational similarity analysis

2.7

From the minimally smoothed single-trialtmaps, we extracted and vectorizedtvalues from each ROI for each trial. Similarity was operationalized as the average Fisher-z-transformed Pearson correlation coefficient between pairs of trials. Our approach is illustrated inFigure 1C. For the MTL ROIs (PHC, HC), similarity values were calculated separately for the left and right hemisphere and then averaged. Though we did not expect hemispheric specialization in the MTL (Meyer & Benoit, 2022;H. Schultz, Sommer, et al., 2022;H. Schultz et al., 2019), this approach rules out that hemispheric differences in ROI size (Pedraza et al., 2004) or in univariate activation drive the observed pattern.

First, to test for episodic reinstatement of scene-specific information, we computedencoding-retrieval-similarity. Here, we computed the average similarity between pairs consisting of an encoding and a retrieval trial, separately for the following conditions: High-reward remembered trial pairs sharing the same scene (HR-same) or different scene (HR-diff), and similarly for high-reward forgotten trial pairs (HF-same, HF-diff), low-reward remembered trial pairs (LR-same, LR-diff), and low-reward forgotten trial pairs (LF-same, LF-diff). We excluded trial pairs containing the same object (mean [range]: 79.99 [28 132] excluded trial pairs per subject and condition). This step was taken to avoid artificially inflating same-scene similarity by including same-object similarity, which may be driven by perceptual input. We also excluded trial pairs associated with incorrect encoding responses. These were trial pairs for which either the encoding trial itself or its repetition during the encoding phase were incorrect (an additional 314.50 [0 2116] trial pairs per subject and condition). Thus, we retained 1378.58 [120 3456] trial pairs per subject and condition.

Second, to test for memory generalization, we computedretrieval-retrieval similarity. Here, we computed the average similarity between pairs of retrieval trials, separately for the following conditions: High-reward remembered trial pairs sharing either the same scene (HR-same) or different scenes (HR-diff), and similarly for high-reward forgotten trials (HF-same, HF-diff), low-reward remembered trials (LR-same, LR-diff), and low-reward forgotten trials (LF-same, LF-diff). We excluded trial pairs from the same fMRI run (mean [range]: 98.17 [8 242] excluded trial pairs per subject and condition) as well as trial pairs associated with incorrect encoding responses. These were trial pairs for which any of the encoding trials were incorrect (an additional 59.57 [0 395] trial pairs per subject and condition). Thus, we retained 262.03 [25 650] trial pairs per subject and condition.

Statistical analyses of the similarity values were conducted in R. To test for effects of our experimental manipulations on encoding-retrieval similarity and retrieval-retrieval similarity in each ROI, we submitted the mean similarity scores for each participant and condition to repeated-measures ANOVAs (R: afex::aov_ez) (Singmann et al., 2021) with the factors reward (high, low), memory (remembered, forgotten), and scene overlap (same scene, different scene). We conducted follow-up paired comparisons as necessary.

Representational similarity analysis—complementary searchlight analyses

2.8

We complemented the above ROI analyses of encoding-retrieval similarity and retrieval-retrieval similarity with a searchlight analysis. To this end, we repeated the above analysis within a moving searchlight (3 voxel radius) centered on every voxel inside each participant’s brain mask. For each condition (i.e., HR-same, HR-diff, HF-same, HF-diff, LR-same, LR-diff, LF-same, LF-diff), the resulting Fisher-z transformed correlation coefficients were written out as a statistical image. For each participant, we then combined these statistical images into first-level contrast images: Following up on the ROI results, for encoding-retrieval similarity, we computed the interaction effect of memory and scene overlap ([1 -1 -1 1 1 -1 -1 1]). For retrieval-retrieval similarity, we computed the main effect of scene overlap ([1 -1 1 -1 1 -1 1 -1]). These contrast images were normalized to standard space using the transformation matrix from the T1w normalization, and smoothed with a Gaussian kernel of 6 mm full width at half maximum (SPM12). The resulting maps were then submitted to a second-level one-samplet-test in SPM12. Multiple comparisons correction was achieved through peak-level family-wise error correction (FWE) within the anatomical masks of the HC and PHC (encoding-retrieval similarity) and the mPFC mask (retrieval-retrieval similarity). Finally, to explore effects outside our ROIs, we also applied FWE correction across the whole brain.

Control analyses

2.9

We computed a number of control analyses to rule out that our findings of retrieval-retrieval similarity were driven by scene reinstatement, and test whether imbalanced numbers of trial pairs could have biased our RSA results.

First, if, in a given ROI, retrieval-retrieval similarity reflects transformed memory representations, rather than a byproduct of scene reinstatement, then these effects of retrieval-retrieval similarity should be greater than corresponding effects of encoding-retrieval similarity. Therefore, in those ROIs that showed significant retrieval-retrieval similarity effects, we additionally tested whether these effects were greater than the corresponding encoding-retrieval similarity effects. For example, if an ROI showed a main effect of scene overlap in the retrieval-retrieval similarity analysis, we computed the average main effect of scene overlap for each participant in the retrieval-retrieval similarity analysis as well as the encoding-retrieval similarity analysis. We then compared these values using pairedt-tests.

Second, if retrieval-retrieval similarity effects are driven by scene reinstatement, these might be correlated across participants. To test this, in those ROIs that showed significant effects of scene overlap on retrieval-retrieval similarity, we computed across-subject correlations of the retrieval-retrieval similarity effects with the corresponding encoding-retrieval similarity effects in the same region.

Similarly, if retrieval-retrieval similarity effects are driven by scene reinstatement, these might be correlatedwithinparticipants. Thus, we tested whether scene reinstatement predicted effects of scene overlap in retrieval-retrieval similarity within subjects. To this end, we first calculated single-trial indices of scene reinstatement: For each retrieval trial, we computed the difference between its average similarity to same-scene versus different-scene encoding trials (only using encoding trials of the same condition: HR, HF, LR, LF). Similarly, for single-trial effects of scene overlap in retrieval-retrieval similarity, we computed the difference between each retrieval trial’s average similarity to same-scene versus different-scene retrieval trials (only using retrieval trials of the same condition but from different runs, as above). Within participants, we then used linear regression to predict effects of overlap in retrieval-retrieval similarity based on single-trial scene reinstatement. Finally, the resulting slopes for each participant were submitted to a group-levelt-test against 0.

Lastly, because participants recall and forget memories at different rates in different conditions, the number of trial pairs in each condition of the RSA varies within and across participants. To rule out that our RSA ROI results could be driven by imbalanced numbers of trial pairs in each bin, we computed permutation analyses, similarly toTompary and Davachi (2017). For each of our central ROI effects, within each participant we created a null distribution of effects based on 10,000 random permutations of the condition labels across the included similarity values. Thus, in each permutation, the number of trial pairs in each condition was the same as for the true effect. We then created the z-score of the true effect for each participant relative to the null distribution of effects. Finally, we tested whether these z-scores were greater than 0 across participants using a one-samplet-test.

For the same reason, we repeated our RSA searchlight analyses using a similar permutation approach (albeit with only 1,000 permutations due to the increased computational demands). In short, for the encoding-retrieval similarity effects (memory x overlap interaction) and retrieval-retrieval similarity effects (main effect of overlap),z-scores for each participant’s true effect compared to their null distribution were computed for every searchlight as above. As in the original searchlight analysis, the resulting participant-specificz-maps were then normalized into MNI space and smoothed at 6 mm before being submitted to a 2nd-level one-sample t-test in SPM12.

Results

3

Behavioral results

3.1

Accuracy on the incidental encoding task was, as intended, near ceiling (Fig. 2), and did not differ for high versus low reward trials (t(39)= 0.766,p= 0.449). There was a trend for correct responses to be faster in the high than the low reward condition (t(39)= 1.755,p= 0.087,Fig. 2).

Behavioral results. Left panel: Accuracy during the incidental encoding task. Middle panel: Median response times (RTs) for correct responses during the incidental encoding task. Right panel: Accuracy during the scene recall task. Error bars indicate 95% confidence interval. n.s. indicates not significant, ~ indicates 0.05 <p< 0.1.

Participants correctly recalled around half of the scenes. While accuracy was numerically higher for high-reward trials (high reward: 53.7%, low reward: 51.3%,Fig. 2), the difference was not significant (t(39)= 1.147,p= 0.258).

Encoding-retrieval similarity: The PHC reinstates scene-specific patterns during retrieval

3.2

We tested whether, during retrieval, the three ROIs would reinstate scene-specific patterns from the encoding phase. To this end, we calculated the average similarities between pairs of retrieval trials and encoding trials that either shared the same scene or not (same-scene vs. different-scene similarity). Reinstatement would be reflected in greater same-scene than different-scene similarity. We expected this effect of scene overlap for remembered trials, particularly in the PHC and HC, with the mPFC serving as control.

ROI analyses

3.2.1

In each of the three ROIs (mPFC, PHC, and HC), we computed a three-way repeated-measures ANOVA with the factors reward (high, low), memory (remembered, forgotten), and scene overlap (same scene, different scene). We report main effects of scene overlap as well as interactions that include the scene overlap factor, as other effects (such as, e.g., a main effect of memory) do not reflect reinstatement of the specific scene. The complete results tables are shared on OSF alongside the analysis script (https://osf.io/yracf/).

The mPFC (Fig. 3, left panel) did not show an effect involving the scene overlap factor (allF(1,39)≤ 0.482, all*p *≥ 0.492).

Encoding-retrieval similarity. Average encoding-retrieval similarity values in the three a-priori ROIs: mPFC, PHC, and HC. Notes refer to effects from three-way repeated-measures ANOVAs with the factors reward, memory, and scene overlap (see main text for details). Error bars indicate 95% confidence interval. HR: high-reward/remembered; HF: high-reward/forgotten; LR: low-reward/remembered, LF: low-reward/forgotten; same: same-scene similarity, diff: different-scene similarity;R−same¯,R−diff¯: remembered/same-scene similarity and remembered/different-scene similarity averaged across the reward factor,rrefers to mean Fisher-z transformed Pearson correlation coefficients. The zoomed-in panel clarifies the direction of the interaction effect. * indicatesp< 0.05.

The PHC (Fig. 3, middle panel) showed an interaction of scene overlap and memory (F(1,39)= 5.370,p= 0.026), reflecting a greater scene overlap effect for remembered than forgotten trials. No other effects involving the scene overlap factor were significant (allF(1,39)≤ 2.079, all*p *≥ 0.157). To follow up on this interaction, we averaged over the reward factor, and compared same versus different scene similarity separately for remembered and forgotten trials. Same-scene similarity was significantly greater than different-scene similarity for remembered (t(39)= 2.796,p= 0.008) but not forgotten trials (t(39)= 0.617,p= 0.541). This pattern is consistent with reinstatement of scene-specific encoding activity during successful memory retrieval.

Overall, the HC (Fig. 3, right panel) did not show any effects involving the scene overlap factor (allF(1,39)≤ 1.235, allp ≥ 0.273). However, previous research suggests that the posterior HC may be particularly involved in representing episodic detail (Poppenk et al., 2013). We, therefore, repeated the above analysis separately for the anterior and posterior HC (Supplementary Fig. S1). The anterior HC did not show any effects involving the scene overlap factor (allF(1,39)≤ 0.367, allp ≥ 0.548), while the posterior HC only showed a trend-level interaction of reward and scene overlap (F(1,39)= 3.534,p= 0.068, all otherF(1,39)≤ 0.891, all*p *≥ 0.351, numerically greater effect of scene overlap for low-reward than high-reward trials).

Complementary searchlight analysis

3.2.2

To follow up on our findings of scene-specific reinstatement during remembered trials in the PHC, we repeated the same analysis using a searchlight approach. The contrast for the interaction effect of memory and scene overlap (i.e., [1 -1 -1 1 1 -1 -1 1] on the conditions HR-same, HR-diff, HF-same, HF-diff, LR-same, LR-diff, LF-same, LF-diff) was computed on the single-subject level and submitted to a one-samplettest on the group level. However, atp< 0.001 uncorrected, this analysis yielded no significant voxels within a combined mask of the PHC and HC. Furthermore, no voxels survived FWE correction across the whole brain. We note that, due to the high interindividual variability in MTL anatomy (Pruessner et al., 2002), analyses in group space are typically less sensitive than analyses within individual MTL ROIs. The unthresholdedtmap may be found on OSF (https://osf.io/yracf/).

Retrieval-retrieval similarity: The mPFC and anterior HC represent transformed memories

3.3

Next, we tested whether memory representations with overlapping features (i.e., the same scene) showed evidence for generalization. To this end, we calculated the average similarities between pairs of retrieval trials that either shared the same scene (same-scene similarity) or not (different-scene similarity). Memory generalization would be reflected in greater same-scene than different-scene similarity. We expect this effect of scene overlap predominantly in the mPFC and HC, with the PHC serving as control.

ROI analyses

3.3.1

As with the encoding-retrieval similarity analysis above, we computed, for each of the ROIs, a three-way repeated-measures ANOVA with the factors reward (high, low), memory (remembered, forgotten), and scene overlap (same scene, different scene). We report main effects of similarity as well as interactions that include the scene overlap factor, as other effects (such as a main effect of memory) do not reflect generalization. The complete results tables are shared on OSF alongside the analysis script (https://osf.io/yracf/).

The mPFC (seeFig. 4A) showed a significant main effect of scene overlap (same-scene similarity > different-scene similarity,F(1,39)= 13.455,p< 0.001). This is consistent with the emergence of generalized memory representations for episodes that share overlapping features. No other effect involving the scene overlap factor was significant (allF(1,39)≤ 1.932, all*p *≥ 0.172).

Retrieval-retrieval similarity. (A) Average retrieval-retrieval similarity values in the three a-priori ROIs: mPFC, PHC, and HC. (B) Average retrieval-retrieval similarity in the anterior versus posterior portion of the HC. (C) Results of whole-brain searchlight analysis for the main effect of scene overlap (same-scene similarity > different-scene similarity across levels of reward and memory). Average retrieval-retrieval similarity was extracted for visualization. Display thresholdp< 0.001 unc.,k= 5 voxels. Notes refer to effects from three-way repeated-measures ANOVAs with the factors reward, memory, and scene overlap (see main text for details). Error bars indicate 95% confidence interval. HR: high-reward/remembered; HF: high-reward/forgotten; LR: low-reward/remembered, LF: low-reward/forgotten; same: same-scene similarity, diff: different-scene similarity;high−same¯,high−diff¯: high reward/same-scene similarity and high reward/different-scene similarity averaged across the memory factor,rrefers to mean Fisher-z transformed Pearson correlation coefficients. The zoomed-in panel clarifies the direction of the interaction effect. * indicatesp< 0.05, ~ indicates 0.05 <p< 0.1.

The PHC (seeFig. 4A) did not yield an effect involving the scene overlap factor (allF(1,39)≤ 0.774, all*p *≥ 0.384).

Overall, for the HC (seeFig. 4A), we observed a trend-level main effect of scene overlap (same-scene similarity > different-scene similarity,F(1,39)= 3.670,p= 0.063), qualified by a trend-level interaction of reward and scene overlap (F(1,39)= 2.891,p= 0.097). This pattern reflected a larger effect of scene overlap for high-reward than low-reward trials. Given that the anterior portion of the HC may be particularly involved in processing generalized information as well as reward (Guo & Yang, 2020;Poppenk et al., 2013), we repeated the above analyses in anterior versus posterior portions of the HC (Fig. 4B).

The anterior HC showed a significant interaction of reward and scene overlap (F(1,39)= 9.743,p= 0.003). This is consistent with reward-enhanced generalization of overlapping memories. No other effect that included the scene overlap factor was significant (allF(1,39)≤ 1.213, all*p *≥ 0.278). To follow up on this interaction, we averaged over the memory factor, and compared same-scene versus different-scene similarity, separately for high-reward and low-reward trials. Same-scene similarity was significantly greater than different-scene similarity for high-reward trials (t(39)= 2.686,p= 0.011), but not for low-reward trials (t(39)= 1.624,p= 0.112). The anterior HC, thus, showed a result pattern that was more pronounced than the result pattern in the whole HC ROI.

The posterior hippocampus, on the other hand, showed a trend-level main effect of scene overlap (same-scene similarity > different-scene similarity,F(1,39)= 3.880,p= 0.056). No other effect involving the scene overlap factor was significant (allF(1,39)≤ 1.652, all*p *≥ 0.206).

Complementary searchlight analysis

3.3.2

To corroborate our main findings of overall memory generalization in the mPFC, we conducted the same analysis using a searchlight approach. The contrast for the main effect of scene overlap (i.e., [1 -1 1 -1 1 -1 1 -1] on the conditions HR-same, HR-diff, HF-same, HF-diff, LR-same, LR-diff, LF-same, LF-diff) was computed on the single-subject level and submitted to a one-sample t-test on the group level. We applied small-volume correction across the anatomical mPFC mask. This analysis yielded a significant peak within the mPFC (MNI coordinates: [6 54 -6],t(39)= 5.212,pSVC= 0.003,Fig. 4C). We visualized this effect by extracting the mean similarity values across the entire cluster (thresholded atp< 0.001) for each condition and subject. The result pattern resembles the one reported for the anatomical mPFC ROI (see above). In addition, two further peaks survived FWE-correction across the whole brain: The left postcentral gyrus ([-44 -22 54,t(39)= 8.890,pFWE≤ 0.001), and the bilateral occipital cortex ([0 -88 2],t(39)= 8.325,pFWE≤ 0.001). The unthresholdedtmap may be found on OSF (https://osf.io/yracf/).

Control analyses

3.4

Effects of retrieval-retrieval similarity in the mPFC and anterior HC go beyond scene reinstatement

3.4.1

It is possible that effects of scene overlap on retrieval-retrieval similarity may be driven by scene reinstatement: If the same-scene pattern is reinstated in two retrieval trials (encoding-retrieval similarity), these trials may be more similar to each other for this reason alone. To test whether the retrieval-retrieval similarity effects in the mPFC and anterior HC go beyond mere scene reinstatement, we therefore tested whether they were greater than the encoding-retrieval similarity effects in the respective region. Therefore, for each of these ROIs, we contrasted the retrieval-retrieval similarity effects with the corresponding encoding-retrieval similarity effect using pairedt-tests.

Indeed, the retrieval-retrieval similarity effects significantly exceeded the corresponding encoding-retrieval similarity effects in both regions. In the mPFC, the retrieval-retrieval similarity main effect of overlap was greater than the encoding-retrieval similarity main effect of overlap (t(39)= 3.160,p= 0.003). In the anterior HC, the retrieval-retrieval similarity interaction effect of reward and overlap was likewise greater than the encoding-retrieval similarity interaction effect of reward and overlap (t(39)= 3.044,p= 0.004). Hence, in either region, it is unlikely that the retrieval-retrieval similarity effects were driven by scene reinstatement.

Effects of retrieval-retrieval similarity are not predicted by scene reinstatement across or within participants

3.4.2

If effects of scene overlap on retrieval-retrieval similarity are driven by scene reinstatement, these may correlate across or within subjects. To this end, in our ROIs that showed significant effects of scene overlap on retrieval-retrieval similarity (mPFC and anterior HC), we tested whether the observed effects on retrieval-retrieval similarity were correlated across subjects with the corresponding effects on encoding-retrieval similarity. In the mPFC, the correlation between the main effect of scene overlap on retrieval-retrieval similarity (average same-scene similarity minus average different-scene similarity) and the corresponding encoding-retrieval similarity effect was numerically positive, but not significant (r= 0.113,p= 0.489). In the anterior HC, the correlation between the interaction effect of reward and scene overlap on retrieval-retrieval similarity and the corresponding effect on encoding-retrieval similarity was also numerically positive, but not significant (r= 0.161,p= 0.322).

Furthermore, if effects of scene overlap on retrieval-retrieval similarity are driven by scene reinstatement, these effects may be correlated within subjects. Hence, we tested whether single-trial reinstatement predicted single-trial effects of scene overlap on retrieval-retrieval similarity. In the mPFC, the linear slopes were numerically positive (mean [std] slope: 0.013 [0.054]), but not significantly different from 0 (t(39)= 1.497,p= 0.142). In the anterior HC, the slopes were numerically negative (mean [std] slope: -0.009 [0.053]), and not significantly different from 0 (t(39)= 1.067,p= 0.293). Given that the anterior HC originally showed an effect of scene overlap on retrieval-retrieval similarity for the high-reward condition, we repeated this analysis using high-reward trials only. The effect remained not significant (mean [std] slope: -0.020 [0.088], (t(39)= 1.427,p= 0.162)).

In sum, scene reinstatement did not predict effects of scene overlap on retrieval-retrieval similarity either across or within subjects, making it unlikely that the former drove the latter effects.

RSA effects are not driven by imbalanced trial numbers

3.4.3

To test whether our central ROI effects in the PHC (encoding-retrieval similarity), mPFC, and anterior HC (retrieval-retrieval similarity) could have been driven by imbalanced numbers of trial pairs across conditions and participants, we computed permutation analyses (see Methods for details).

Importantly, in all three ROIs, the true effects were significantly greater than the null distribution of effects, which was based on identical trial numbers but randomly permuted condition labels. In the PHC, the interaction effect of memory and overlap in the encoding-retrieval similarity analysis was significant (t(39)= 2.400,p= 0.021). In the mPFC, the main effect of overlap in the retrieval-retrieval similarity analysis was significant (t(39)= 3.851,p< 0.001). Finally, in the anterior HC, the interaction effect of reward and overlap in the retrieval-retrieval similarity analysis was also significant (t(39)= 2.987,p= 0.005). Thus, our ROI findings are not due to imbalanced numbers of trial pairs, as any bias due to imbalanced numbers would have been present in the null distributions as well.

Finally, we also repeated our RSA searchlight findings using permutation testing. For the analysis of encoding-retrieval similarity (memory x overlap interaction effect), as in our original analysis, we did not observe significant voxels within a combined mask of the bilateral HC and PHC atp< 0.001 uncorrected. However, outside this mask, one cluster in the left supramarginal gyrus survived peak-level FWE correction (MNI coordinates: [-64 -26 34],t(39)= 5.568,pFWE= 0.044). For the analysis of retrieval-retrieval similarity (main effect of overlap), results were highly similar to the original analysis. Applying small volume correction within the anatomical MNI mask of the mPFC yielded a significant peak very close to the original peak ([6 56 -4],t(39)= 5.231,pSVC= 0.003). Similarly, two further peaks survived FWE-correction across the whole brain: The left postcentral gyrus ([-44 -20 56,t(39)= 8.627,pFWE≤ 0.001), and the bilateral occipital cortex ([0 -88 2],t(39)= 7.981,pFWE≤ 0.001). In sum, we could replicate our RSA searchlight findings.

Additional control analyses

3.4.4

Given that reward effects on encoding responses and memory were not significant, we computed additional analyses, which are included in the Supplemental Information. In short, our study was sufficiently powered to replicate the reward effect on associative memory fromGruber et al. (2016), and our reward manipulation successfully engaged regions of the mesolimbic reward system (seeSupplementary Fig. S2).

Discussion

4

With the present fMRI study, we set out to examine whether the brain concurrently represents episodic memory representations as well as more generalized memory representations that encode commonalities across episodes.

First, we examined the reinstatement of scene patterns from the encoding experience, thought to reflect episodic memory. We observed evidence for memory reinstatement in the PHC, but not the mPFC or HC. Second, we tested for memory transformation—that is, the representational convergence of neural retrieval patterns for memories that share overlapping features. This is thought to reflect the shift from individual episodes into generalized memory. We found evidence for such memory transformation in the mPFC and HC, but not the PHC. Intriguingly, memory transformation in the mPFC affected all memories, regardless of reward magnitude or retrieval success. In contrast, memory transformation in the anterior HC was enhanced by reward. We note, however, that unlike previous studies (e.g.,Gruber et al., 2016;Schultz, Yoo, et al., 2022;Wittmann et al., 2005) we did not observe a reward effect on behavioral retrieval accuracy, and only a marginal effect on response times during encoding (see alsoSchultz et al., 2023). This constrains the conclusions that may be drawn from our findings of reward-enhanced neural similarity in the anterior HC, as their role for behavior remains unclear.

Our results thus extend our knowledge derived from studies that had tested retrieval after a consolidation period of 3 days (Audrain & McAndrews, 2022) and 1 week (Tompary & Davachi, 2017). Here, we demonstrate (i) that memory transformation has already taken place after 1 day, (ii) that it is partly enhanced by reward, and (iii) that generalized memory representations can be activated even in absence of successful retrieval of a particular episodic memory.

Importantly, did our analysis of retrieval-retrieval similarity truly reflect transformed memories? Transformation implies two things: One, that memory representations change over time. Previous studies have focused on this and demonstrated that memories sharing the same scene become more similarto each otherover time (Audrain & McAndrews, 2022;Tompary & Davachi, 2017). Two, such change over time implies that the transformed memory trace will be less similarto itself, that is, to the original encoding pattern. Testing this is equally critical. This is because retrieval-retrieval similarity by itself could also be a consequence of common scene reinstatement: If the same-scene-specific encoding pattern is reinstated in two retrieval trials, these would also be similar to each other. Such an effect would not reflect a transformation away from the original encoding pattern.

To address this, we analyzed concurrent scene reinstatement (i.e., scene-specific encoding-retrieval similarity). Our results imply that retrieval-retrieval similarity was not merely driven by scene reinstatement: First, there was little topographical overlap between the two effects, with scene reinstatement predominantly in the PHC, and retrieval-retrieval similarity predominantly in the mPFC and (anterior) HC. Second, scene reinstatement was modulated by memory success whereas retrieval-retrieval similarity was not. Third, effects of retrieval-retrieval similarity in the mPFC and anterior HC were significantly greater than the corresponding effects of encoding-retrieval similarity, indicating that they were unlikely to be driven by scene reinstatement. Fourth, scene reinstatement did not predict effects of retrieval-retrieval similarity in these regions, either across or within participants. This pattern indicates that our findings in the mPFC and anterior HC truly reflect a transformed memory representation rather than a reinstatement of a shared encoding pattern.

Memory transformation—in the sense of representational convergence of overlapping memories—has previously been observed for the mPFC (Audrain & McAndrews, 2022;Tompary & Davachi, 2017). These findings are consistent with a role of the mPFC in representing generalized knowledge structures (Ghosh & Gilboa, 2014;Gilboa & Marlatte, 2017;Morrissey et al., 2017;Paulus et al., 2021).

Generalization may be driven by replay of episodic memories during consolidation. This process would allow the cortex to extract commonalities across similar memories and to store these as more generalized representations (Kumaran et al., 2016;Liu et al., 2019;Sekeres et al., 2018). Here, we show that such effects do not require 3 (Audrain & McAndrews, 2022) or 7 days (Tompary & Davachi, 2017) to emerge. Instead, they are already present a single day after encoding.

Notably, in the mPFC as well as the anterior HC, the activation of transformed memory representations was independent of memory success. This appears to contradict previous studies that reported memory transformation for correctly recalled trials only (Audrain & McAndrews, 2022;Tompary & Davachi, 2017). Both of these studies included control analyses for trials without overt scene recall (recognition trials,Tompary & Davachi, 2017; forgotten trials,Audrain & McAndrews, 2022). In both cases, the mPFC patterns were numerically consistent with representational convergence, similar to our findings, though they were statistically inconclusive. Transformed memories in the HC, on the other hand, appeared more dependent on overt memory success (Audrain & McAndrews, 2022;Tompary & Davachi, 2017), which is more at odds with our findings. If the mPFC encodes representations that generalize across episodes that share common content (Audrain & McAndrews, 2022;Gilboa & Marlatte, 2017;Sekeres et al., 2018), these would get activated whenever one of these episodes is being probed. However, given that these representations abstract away from unique features that are specific to individual episodes, they would not contain sufficient episodic detail to drive episodic recall. In the case of the HC, it is possible that our shorter delay of 1 day contributed to our findings. One could speculate that also memories of forgotten trials contained more information on the scene context than in previous studies (Audrain & McAndrews, 2022;Tompary & Davachi, 2017). This would make retrieval-retrieval similarity less dependent on overt memory success.

Contrary to our hypothesis, reward did not foster memory transformation in the mPFC, but only in the anterior HC. Reward has been shown to increase neural replay (Gruber et al., 2016;Sterpenich et al., 2021) and promote consolidation (Murayama & Kitagami, 2014;Murayama & Kuhbandner, 2011;Spaniol et al., 2014;Wittmann et al., 2005). Therefore, we had hypothesized that reward would facilitate representational convergence. The reason for this dissociation between the mPFC and anterior HC is unclear. The mPFC has been previously demonstrated to encode value-shaped representations of knowledge (Baram et al., 2021;Moneta et al., 2023;Paulus et al., 2021). Furthermore, both the mPFC and HC are linked to the brain’s reward circuit (Haber & Knutson, 2010), and the anterior portion of the HC is particularly connected to the mPFC (Adnan et al., 2016;Barnett et al., 2021;Poppenk et al., 2013). Specifically, post-encoding functional connectivity between the mPFC and anterior HC predicts subsequent memory transformation, both behaviorally (Audrain & McAndrews, 2022) and neurally (Tompary & Davachi, 2017). The anterior HC may also be particularly involved in motivational aspects of memory (Murty et al., 2017;Poppenk et al., 2013). Hence, one may have expected similar effects of reward on memory transformation in the anterior HC and mPFC.

However, consolidation is not complete after one night. Given that memory transformation may last for years, accompanied by a neural shift from HC to neocortex (Sekeres et al., 2018), it is possible that such reward effects on neural similarity emerge first in the HC and then shift to the mPFC at a later time point. Indeed, the HC may constitute a quick learning system that rapidly acquires not only episodic memory traces, but also regularities from similar events (Kumaran & McClelland, 2012;Schapiro et al., 2017). Through reward-biased replay, it may then coordinate the acquisition of generalized memory representations in the slower neocortical system (Kumaran et al., 2016). This may be one reason why, after a comparatively short time window of 1 day, we only observed reward effects on memory transformation in the anterior HC. Whether the differences between the mPFC and HC, indeed, reflect different time-courses of reward-enhanced memory transformation could be tested in future work using repeated retrieval sessions.

Other regions beyond the MTL and mPFC may contribute to consolidation and ultimately transformation of high-reward memories, however. For example,Murty et al. (2017)found that, post-encoding, the functional connectivity between category-selective visual cortex and the anterior HC as well as the dopaminergic midbrain was selectively associated with memory for high-reward associations. In line with our considerations above, it is possible that the anterior HC, rather than the mPFC, predominantly orchestrates brain-wide processes underlying reward-enhanced memory transformation, at least at the time-point examined here.

The hippocampus has been suggested to be functionally differentiated along its longitudinal axis, with more gist-like, schematic representations in the anterior HC, and more fine-grained, episodic representations in the posterior HC (Collin et al., 2015;Gilboa & Moscovitch, 2021;Guo & Yang, 2020;Poppenk et al., 2013;Sekeres et al., 2018), although seeTompary and Davachi (2017). For example,Audrain and McAndrews (2022)reported that the anterior HC distinguishes the general context of retrieved memories (i.e., beach or kitchen), whereas the posterior hippocampus distinguishes specific details of these contexts (i.e., the specific beach or kitchen). Our results only partially support such a distinction. We, indeed, observed distinct result patterns along the longitudinal axis of the HC: The anterior HC showed increased memory transformation for high-reward retrieval trials. The posterior HC, on the other hand, did not yield evidence for episodic scene reinstatement.

Instead, we only observed scene reinstatement in the PHC. When participants correctly recalled a scene, the activation pattern in the PHC was more similar to the activation pattern during encoding of that scene. This is in line with previous research showing scene-specific pattern reinstatement in the PHC (Meyer & Benoit, 2022;H. Schultz et al., 2019;H. Schultz, Sommer, et al., 2022;Staresina et al., 2012) and scene-specific memory processing in general (Liang & Preston, 2017;H. Schultz et al., 2012;H. Schultz, Yoo, et al., 2022;Staresina et al., 2013). Anatomically, the PHC is a connecting hub between the dorsal visual stream and downstream regions in the MTL, including the entorhinal cortex and HC (Lavenex & Amaral, 2000;Suzuki & Amaral, 1994a,1994b), and thus well-positioned to support spatial, scene-specific, or contextual memory (Eichenbaum et al., 2007). We note that our findings of scene reinstatement reflect patterns that are common to a range of memories associated with the same scenes, rather than event-unique patterns. They may, therefore, even contain some aspects of memory generalization that may already started to occur during the encoding phase.

The concurrent presence of original and transformed memory representations is consistent with accounts that the same memories exist in multiple forms at different levels of abstraction, with their relative strength of activation dependent on, for example, task demands (Gilboa & Moscovitch, 2021;Sekeres et al., 2018). Here, we observed that the same retrieval trials elicited both reinstated and transformed memory representations. These were present in the MTL and mPFC, respectively. Does this suggest that the two are independent of each other? Previous work has demonstrated that, during autobiographical retrieval, mPFC activity precedes and drives HC activity (McCormick et al., 2020, but seeCampbell et al., 2018). Furthermore, a higher integrity of the anatomical connection between the HC and mPFC is associated with richer autobiographical memories (Williams et al., 2020). It is possible that the memory representations in the mPFC are instantiated earlier, but are by themselves insufficient to elicit successful episodic retrieval. However, if these are passed on through top-down modulation of the MTL (McCormick et al., 2020;Nawa & Ando, 2019;St Jacques et al., 2011), they may guide episodic retrieval and thus aid in recovering the details of a memory trace. Other methods with a higher temporal resolution, such as electrophysiological measures, may further elucidate these potential interactions between the complementary memory representations in the mPFC and MTL.

Lastly, our whole-brain searchlight analysis also revealed a visual cortex cluster showing greater retrieval-retrieval similarity for overlapping than non-overlapping memories.Tompary and Davachi (2017)reported a highly similar effect. They propose that this region may represent specific features of memories during retrieval. However, given that we did not observe evidence for visual cortex involvement in more veridical memory representations (encoding-retrieval similarity), we suggest that other factors may play a role. For example, eye movements could be more similar during retrieval of the same versus different scenes.

We note some limitations of the present study. First, we only tested delayed, but not recent retrieval.Tompary and Davachi (2017)tested both. They found relatively greater retrieval-retrieval similarity for overlapping than non-overlapping memories in the anterior HC; however, this was not driven by an increase of overlapping similarity over time, but by a decrease in non-overlapping similarity. We cannot test this in our data, and our findings could be consistent with either of these observations.

As a second limitation, unlike in previous reports (e.g.,Gruber et al., 2016;H. Schultz, Yoo, et al., 2022;Wittmann et al., 2005), our reward manipulation did not enhance memory. While our study was highly powered to replicate the behavioral reward effect fromGruber et al. (2016, seeSupplemental Information), we did introduce some changes to their procedure. One, we randomized reward over trials, rather than presenting it blockwise; however, this should not have affected the reward effect on memory (e.g.,Schultz, Yoo, et al., 2022). Second, we tested retrieval the next, rather than the same day; however, this could potentially even increase the reward effect on memory (Murayama & Kitagami, 2014;Patil et al., 2017). Third, we presented each object-scene pair twice. This was done in order to boost next-day recall success (given rather low memory performance inGruber et al., 2016) and foster generalization (Tompary & Davachi, 2017). In a previous report on a parallel behavioral study (H. Schultz et al., 2023), we argued that such repetition could potentially decrease the mesolimbic response through decreasing the reward prediction error (Rouhani et al., 2018;Rouhani & Niv, 2021;W. Schultz, 1998) and reducing the novelty of the object-scene pair (Bunzeck & Düzel, 2006;Bunzeck et al., 2012;Krebs et al., 2009;Wittmann et al., 2007). However, note that we did observe a mesolimbic reward response (Supplementary Fig. S2). Lastly, repetition could lead to stronger encoding; however, it is weakly encoded items that show the strongest reward effects (Shigemune et al., 2017;Yan et al., 2022). Future research may further delineate the circumstances under which reward boosts memory.

In summary, we have provided evidence for the concurrent activation of two types of memory representations—in the PHC of the original activity pattern that was present during encoding and in the mPFC and anterior HC of activity patterns that resemble transformed, generalized memories. Reward enhances neural memory transformation in the anterior HC, though it does not reliably promote episodic memory. Our results, thus, broaden our knowledge of the processes that lead to memory generalization, while motivating new questions about how reward shapes the structure of memory.

Supplementary Material

Supplementary Material