Increasing evidence shows that, like phenotypic mean, phenotypic variance is also genetically determined, but the underlying mechanisms of genetic control over the variance remain obscure. Here, we conducted variance-association mapping analyses to identify expression variability QTLs (evQTLs), i.e., genomic loci associated with gene expression variance, in humans. We discovered that common genetic variations may contribute to increasing gene expression variability via two distinct modes of action--epistasis and destabilization. Specifically, the epistasis model explains a quarter of the identified evQTLs, of which the formation is attributed to the presence of "third-party" eQTLs that influence the level of gene expression in a fraction, rather than the entire set, of sampled individuals. The destabilization model explains the other three-quarters of evQTLs, which tend to be associated with mutations that disrupt the stability of the transcription process of genes. To show the destabilizing effect, we measured discordant gene expression between monozygotic twins, and time-course stability of gene expression in single samples using repetitive qRT-PCR assays. The destabilizing evQTL SNPs were found to be associated with more pronounced expression discordance between twin pairs and less stable gene expression in single samples. Together, our results suggest that common SNPs may work interactively or independently to shape the variability of gene expression in humans. These findings contribute to the understanding of the mechanisms of genetic control over phenotypic variance and may have implications for the development of variability-centered analytic methods for quantitative trait mapping.