Decades of research have investigated the impact of clinical depression on memory, which has revealed biases and in some cases impairments. However, little is understood about the effects of sub-clinical symptoms of depression on memory performance in the general population. Here we report the effects of symptoms of depression on memory problems in a large population-derived cohort (N = 2544), 87% of whom reported at least one symptom of depression. Specifically, we investigate the impact of depressive symptoms on subjective memory complaints, objective memory performance on a standard neuropsychological task and, in a subsample (n = 288), objective memory in affective contexts. There was a dissociation between subjective and objective memory performance, with depressive symptoms showing a robust relationship with self-reports of memory complaints, even after adjusting for age, gender, general cognitive ability and symptoms of anxiety, but not with performance on the standardised measure of verbal memory. Contrary to our expectations, hippocampal volume (assessed in a subsample, n = 592) did not account for significant variance in subjective memory, objective memory or depressive symptoms. Nonetheless, depressive symptoms were related to poorer memory for pictures presented in negative contexts, even after adjusting for memory for pictures in neutral contexts. Thus the symptoms of depression, associated with subjective memory complaints, appear better assessed by memory performance in affective contexts, rather than standardised memory measures. We discuss the implications of these findings for understanding the impact of depressive symptoms on memory functioning in the general population.