Abstract
What makes some images memorable while others are forgettable? The features of an image can be represented at multiple levels – from low-level visual properties to high-level meaning. Across two behavioral studies and a neuroimaging study, we addressed the question of how image memorability is influenced by different levels of the visual hierarchy. In a first behavioral study, we combined a convolutional neural network (CNN) with behavioral prospective assignment, by using one of four CNN layers to select the scene images that each of one hundred participants experience. We found that participants remembered more images when they were assigned to view stimuli that were identified as discriminable using low-level CNN layers, or identified as similar in high-level layers. A second study replicated the first experiment’s results using images from a single semantic category (houses), but found that similarity predicted memorability at a slightly less high-level that holds representations of objects, suggesting this level is more important for remembering images from the same category. Finally, we analyzed neural activity collected through functional magnetic resonance imaging (fMRI) scans as independent participants viewed the same scene images. Pattern similarity analyses revealed an analogous relationship in the ventral stream between image discriminability/similarity and level of the visual hierarchy. Discriminability in early visual areas, and similarity later in the ventral stream, each predicted greater image memorability. Together, this research shows that discriminability at different visual levels can be used to predict image memorability through both CNN models and neural activity in the human ventral stream.