A substantial portion of intraspecific diversity is associated with local adaptation to environment, which is driven by genotype-by-environment interactions (GxE) for fitness. Local adaptation is often studied via 1) multiple common garden experiments comparing performance of genotypes in different environments and 2) sequencing genotypes from multiple locations and characterizing geographic patterns in allele frequency. Both approaches aim to characterize the same pattern (local adaptation), yet the complementary information from each approach has not been coherently integrated into a modeling framework. Here, we develop a genome-wide association model of genotype interactions with continuous environmental gradients (GxE). We employ an imputation approach to synthesize evidence from common garden and genome-environment associations, allowing us to identify loci exhibiting environmental clines where alleles are associated with higher fitness in home environments. We apply this model to simulations and published data on Arabidopsis thaliana. Simulations showed our approach increases power to detect loci causing local adaptation. In Arabidopsis, our approach revealed candidate genes for local adaptation based on known involvement in environmental stress response. Most identified SNPs exhibited home allele advantage and fitness tradeoffs along climate gradients, suggesting selective gradients maintain allelic clines. SNPs exhibiting GxE associations with fitness were enriched in genic regions, putative partial selective sweeps, and GxE associations with an adaptive phenotype (flowering time). We discuss extensions for situations where only adaptive phenotypes other than fitness are available. Many types of data may point toward the loci underlying GxE and local adaptation; coherent models of these diverse data provide a principled basis for synthesis.