Multiple genetic and environmental factors contribute to metabolic disease, with effects that range across molecular, organ, and whole-organism levels. Dissecting this multi-scale complexity requires systems genetics approaches to infer polygenic networks that influence gene expression, serum biomarkers, and physiological measures. In recent years, multi-parent model organism crosses, such as the Diversity Outbred (DO) mice, have emerged as a powerful platform for such systems approaches. The DO mice harbor extensive phenotypic and genetic diversity, allowing for detection of multiple quantitative trait loci (QTL) and their interactions at high genomic resolution. In this study, we used 474 DO mice to model genetic interactions influencing hepatic transcriptome expression and physiological traits related to metabolic disease. Body composition, serum biomarker, and liver transcriptome data from mice fed either a high-fat or standard chow diet were combined and simultaneously modeled. Modules of co-expressed transcripts were identified with weighted gene co-expression network analysis, with summary module phenotypes representing coordinated transcriptional programs linked to specific biological functions. We then used the Combined Analysis of Pleiotropy and Epistasis (CAPE) to simultaneously detect directed epistatic interactions between haplotype-specific QTL for transcript modules and physiological phenotypes. By combining information across multiple phenotypic levels, we identified networks of QTL with numerous interactions that reveal how genetic architecture affects metabolic traits at multiple scales. Specifically, these networks model how gene regulatory programs from different inbred founder strains influence more complex physiological traits. By connecting three levels of the organismal hierarchy — genetic variation, transcript abundance, and physiology — we revealed a detailed picture of genetic interactions influencing complex traits through differential gene expression.