Sexual dimorphism manifests in many diseases and may drive sex-specific therapeutic responses. To understand the molecular basis of sexual dimorphism, we conducted a comprehensive assessment of gene expression and regulatory network modeling in 31 tissues using 8716 human transcriptomes from GTEx. We observed sexually dimorphic patterns of gene expression involving as many as 60% of autosomal genes, depending on the tissue. Interestingly, sex hormone receptors do not exhibit sexually dimorphic expression in most tissues; however, differential network targeting by hormone receptors and other transcription factors (TFs) captures their downstream sexually dimorphic gene expression. Furthermore, differential network wiring was found extensively in several tissues, particularly in brain, in which not all regions exhibit strong differential expression. This systems-based analysis provides a new perspective on the drivers of sexual dimorphism, one in which a repertoire of TFs plays important roles in sex-specific rewiring of gene regulatory networks.