Aging-related declines in vision can decrease well-being of the elder. Concerning early sensory changes as in the primary visual cortex, physiological and behavioral reports seem contradictory. Neurophysiological studies on orientation tuning properties suggested that neuronal changes might come from decreased cortical local inhibition. However, behavioral results either showed no clear deficits in orientation processing in the elder, or proposed stronger surround suppression. Through psychophysical experiments conducted on old and young human subjects combined with computational modeling, we resolved these discrepancies by demonstrating stronger lateral inhibition in the elder while neuronal orientation tuning widths, related to local inhibition, stayed globally intact across age. We confirmed this later finding by re-analyzing published neurophysiological data from rhesus monkeys, which showed no systematic tuning width changes, but instead a higher neuronal noise with aging. These results suggest a stronger lateral inhibition and mixed effects on local inhibition during aging, revealing a more complex picture of age-related effects in the central visual system than previously thought.