Post-translational modifications of histone residue tails are an important component of genome regulation. It is becoming increasingly clear that the combinatorial presence and absence of various modifications define discrete chromatin states which determine the functional properties of a locus. An emerging experimental goal is to compare genome-wide chromatin state maps across different conditions, such as experimental treatments, cell-types or developmental time points. Here we present chromstaR, an algorithm for the computational inference of combinatorial chromatin state dynamics across an arbitrary number of conditions. ChromstaR uses a multivariate Hidden Markov Model to assign every genomic region to a discrete combinatorial chromatin state based on the presence/absence of each modification in every condition. This interpretation makes it easy to relate the inferred chromatin states back to the underlying histone modification patterns. Moreover, the algorithm computes the number of combinatorial chromatin states that are present in the genome without having to specify them a priori, thus providing an unbiased picture of their genome-wide frequencies. We demonstrate the advantages of chromstaR in the context of three common experimental data scenarios. First, we study how different histone modifications combine to form combinatorial chromatin states in a single tissue. Second, we infer genome-wide patterns of combinatorial state differences between two cell types or conditions. Finally, we study the dynamics of combinatorial chromatin states during tissue differentiation involving up to six differentiation points. chromstaR is a versatile computational tool that facilitates a deeper biological understanding of chromatin organization and dynamics. The algorithm is written in C++ and freely availableas an R-package at https://github.com/ataudt/chromstaR.