Recent work has hinted at the linkage disequilibrium (LD) dependent architecture of human complex traits, where SNPs with low levels of LD (LLD) have larger per-SNP heritability after conditioning on their minor allele frequency (MAF). However, this has not been formally assessed, quantified or biologically interpreted. Here, we analyzed summary statistics from 56 complex diseases and traits (average N = 101,401) by extending stratified LD score regression to continuous annotations. We determined that SNPs with low LLD have significantly larger per-SNP heritability. Roughly half of the LLD signal can be explained by functional annotations that are negatively correlated with LLD, such as DNase I hypersensitivity sites (DHS) and histone marks. The remaining signal is largely driven by MAF-adjusted predicted allele age (P = 2.38 x 10-104), with the youngest 20% of common SNPs explaining 3.9x more heritability than the oldest 20% -substantially larger than MAF-dependent effects (1.8x). We also inferred jointly significant effects of other LD-related annotations, including smaller per-SNP heritability for SNPs in high recombination rate regions, opposite to the direction of the LLD effect but consistent with the Hill-Robertson effect. Effect directions were remarkably consistent across traits, but with varying magnitude. Forward simulations confirmed that these findings are consistent with the action of negative selection on deleterious variants that affect complex traits, complementing efforts to learn about negative selection by analyzing much smaller rare variant data sets.