The ability to generate variable movements is essential for learning and adjusting complex behaviors. This variability has been linked to the temporal irregularity of neuronal activity in the central nervous system. However, how neuronal irregularity actually translates into behavioral variability is unclear. Here we combine modeling, electrophysiological and behavioral studies to address this issue. We demonstrate that a model circuit comprising topographically organized and strongly recurrent neural networks can autonomously generate irregular motor behaviors. Simultaneous recordings of neurons in singing finches reveal that neural correlations increase across the circuit driving song variability, in agreement with the model predictions. Analyzing behavioral data, we find remarkable similarities in the babbling statistics of 5-6 month-old human infants and juveniles from three songbird species, and show that our model naturally accounts for these 'universal' statistics.