Understanding how the natural world will be impacted by environmental change is one of the most pressing challenges facing humanity. Addressing this challenge is difficult because environmental change can generate both population level plastic and evolutionary responses, with plastic responses being either adaptive or non-adaptive. We develop an approach that links mechanistic quantitative genetic theory with data-driven structured models to allow prediction of population responses to environmental change via plasticity and adaptive evolution. After introducing general new theory, we construct a number of example models to demonstrate that evolutionary responses to environmental change will be considerably slower than plastic responses, that adaptive plasticity can accelerate population recovery to environmental change but that it slows the rate of adaptation to the new environment. Parameterization of the models we develop requires information on genetic and phenotypic variation and demography which will not always be available. We consequently develop a method based on the statistical analysis of temporal trends in model parameter values of examining whether the full machinery of the evolutionarily explicit models we develop will be needed to predict responses to environmental change, or whether simpler non-evolutionary models that are now widely constructed may be sufficient.