Sperm cooperation has evolved in a variety of taxa and is often considered a response to sperm competition, yet the benefit of this form of collective movement remains unclear. Here we use fine-scale imaging and a minimal mathematical model to study sperm aggregation in the rodent genus Peromyscus. We demonstrate that as the number of sperm cells in an aggregate increase, the group moves with more persistent linearity but without increasing speed; this benefit, however, is offset in larger aggregates as the geometry of the group forces sperm to swim against one another. The result is a non-monotonic relationship between aggregate size and average velocity with both a theoretically predicted and empirically observed optimum of 6-7 sperm/aggregate. To understand the role of sexual selection in driving these sperm group dynamics, we compared two sister-species with divergent mating systems and find that sperm of P. maniculatus (highly promiscuous), which have evolved under intense competition, form optimal-sized aggregates more often than sperm of P. polionotus (strictly monogamous), which lack competition. Our combined mathematical and experimental study of coordinated sperm movement reveals the importance of geometry, motion and group size on sperm velocity and suggests how these physical variables interact with evolutionary selective pressures to regulate cooperation in competitive environments.