Neurons in the hippocampus and adjacent brain areas show a large diversity in their tuning to location and head direction. The underlying circuit mechanisms are not fully resolved. In particular, it is unclear why certain cell types are selective to one spatial variable, but invariant to another. For example, a place cell is highly selective to location, but typically invariant to head direction. Here, we propose that all observed spatial tuning patterns -- in both their selectivity and their invariance -- are a consequence of the same mechanism: Excitatory and inhibitory synaptic plasticity that is driven by the spatial tuning statistics of synaptic inputs. Using simulations and a mathematical analysis, we show that combined excitatory and inhibitory plasticity can lead to localized, grid-like or invariant activity. Combinations of different input statistics along different spatial dimensions reproduce all major spatial tuning patterns observed in rodents. The model is robust to changes in parameters, develops patterns on behavioral time scales and makes distinctive experimental predictions. Our results suggest that the interaction of excitatory and inhibitory plasticity is a general principle for the formation of neural representations.