Reconstructing Spatio-Temporal Trajectories of Visual Object Memories in the Human Brain

Abstract

How the human brain reconstructs, step-by-step, the core elements of past experiences is still unclear. Here, we map the spatio-temporal trajectories along which visual object memories are reconstructed during associative recall. Specifically, we inquire whether retrieval reinstates feature representations in a copy-like but reversed direction with respect to the initial perceptual experience, or alternatively, this reconstruction involves format transformations and regions beyond initial perception. Participants from two cohorts studied new associations between verbs and randomly paired object images and subsequently recalled the objects when presented with the corresponding verb cue. We first analyse multivariate fMRI patterns to map where in the brain high- and low-level object features can be decoded during perception and retrieval, showing that retrieval is dominated by conceptual features, represented in comparatively late visual and parietal areas. A separately acquired EEG dataset is then used to track the temporal evolution of the reactivated patterns using similarity-based EEG-fMRI fusion. This fusion suggests that memory reconstruction proceeds from anterior fronto-temporal to posterior occipital and parietal regions, in line with a conceptual-to-perceptual gradient, but only partly following the same trajectories as during perception. Specifically, a linear regression statistically confirms that the sequential activation of ventral visual stream regions is reversed between image perception and retrieval. The fusion analysis also suggests an information relay to fronto-parietal areas late during retrieval. Together, the results shed light onto the temporal dynamics of memory recall, and the transformations that the information undergoes between the initial experience and its later reconstruction from memory. We combined EEG and fMRI to investigate which features of a visual object are reactivated when recalled from episodic memory, and how the memory reconstruction stream unfolds over time and across the brain. Our findings suggest that relative to perception, memory retrieval follows a backwards information trajectory along a conceptual-to-perceptual gradient, and additionally relays retrieved information to multimodal fronto-parietal brain regions. These findings address the fundamental question of whether memories are more or less truthful reconstructions of past events, or instead are subject to systematic biases that prioritise some types of features over others. Our data suggests that episodic memory retrieval is a dynamic and highly reconstructive process with clear prioritisation of abstract-conceptual over detailed-perceptual information. [Abstract copyright: Copyright © 2024 Lifanov-Carr et al.]