A number of open questions in human evolutionary genetics would become tractable if we were able to directly measure evolutionary fitness. As a step towards this goal, we developed a method to test whether individual genetic variants, or sets of genetic variants, currently influence viability. The approach consists in testing whether the frequency of an allele varies across ages, accounting for variation in ancestry. We applied it to the Genetic Epidemiology Research on Aging (GERA) cohort and to the parents of participants in the UK Biobank. In the GERA cohort, the top signal is the APOE 4 allele (P<10-15), whereas in the UK Biobank, the strongest signals are detected in males only, and are for variants near CHRNA3 (P=4×10-8) as well as set of genetic variants that influence heart disease and lipid levels. We suggest that gene-by-environment interactions have altered the genetic architecture of viability in these two cohorts.