Europe has played a major role in dog evolution, harbouring the oldest uncontested Palaeolithic remains and having been the centre of modern dog breed creation. We sequenced the whole genomes of an Early and End Neolithic dog from Germany, including a sample associated with one of Europe’s earliest farming communities. Both dogs demonstrate continuity with each other and predominantly share ancestry with modern European dogs, contradicting a Late Neolithic population replacement previously suggested by analysis of mitochondrial DNA and a Late Neolithic Irish genome. However, our End Neolithic sample possesses additional ancestry found in modern Indian dogs, which we speculate may be derived from dogs that accompanied humans from the Eastern European steppe migrating into Central Europe. By calibrating the mutation rate using our oldest dog, we narrow the timing of dog domestication to 20,000-40,000 years ago. Interestingly, the extreme copy number expansion of the AMY2B gene found in modern dogs was not observed in the ancient samples, indicating that the AMY2B copy number increase arose as an adaptation to starch-rich diets after the advent of agriculture in the Neolithic period.