The variable spiking discharge of sensory neurons in response to a fixed stimulus tends to be weakly correlated (spike-count correlation, rsc), an observation with profound implications for neural coding of sensory information. However, the source of rsc is unclear. It is widely thought to reflect 'bottom up' stochastic noise in shared sensory afferents. However, it may also be generated by changes over time in feedback from higher-order brain regions. Here we test this alternative directly by measuring spiking activity in populations of primary visual cortical (V1) neurons in rhesus monkeys while the animals performed different visual discrimination tasks on the same set of visual inputs. We found that the structure of rsc (the way rsc varied with neuronal stimulus preference) changed dramatically with task instruction despite identical retinal input, directly implying that rsc structure primarily reflects feedback dynamics engaged by the task, not noise in sensory afferents. These results fundamentally alter our view of the origin and function of rsc in sensory neurons, suggesting that correlated variability may be best described as a signature of neural computation rather than stochastic sensory encoding.