Inference of demography and mutation rates is of major interest but difficult because genetic data is only informative about the population mutation rate, the product of the effective population size times the mutation rate, and not about these quantities individually. Here we show that this limitation can be overcome by combining genetic data with pedigree information. To successfully use pedigree data, however, important aspects of real populations such as the presence of two sexes, unbalanced sex ratios and overlapping generations have to be taken into account. We present here an extension of the classic Wright-Fisher model accounting for these effects and show that the coalescent process under this model reduces to the classic Kingman coalescent with specific scaling parameters. We further derive the probability of a pedigree under that model and show how pedigree data can thus be used to infer demographic parameters. Finally, we present a computationally efficient inference approach combining pedigree information and genetic data summarized by the site frequency spectrum (SFS) that allows for the joint inference of the mutation rate, sex-specific population sizes and the fraction of overlapping generations. Using simulations we then show that these parameters can be accurately inferred from pedigrees spanning just a few generations, as are available for many species. We finally discuss future possible extensions of the model and inference framework necessary for applications to wild and domesticated species, namely the account for more complex demographies and the uncertainty in assigning pedigree individuals to specific generations.