Gene expression heterogeneity in the pluripotent state of mouse embryonic stem cells (mESCs) has been increasingly well-characterized. In contrast, exit from pluripotency and lineage commitment have not been studied systematically at the single-cell level. Here we measured the gene expression dynamics of retinoic acid driven mESC differentiation using an unbiased single-cell transcriptomics approach. We found that the exit from pluripotency marks the start of a lineage bifurcation as well as a transient phase of susceptibility to lineage specifying signals. Our study revealed several transcriptional signatures of this phase, including a sharp increase of gene expression variability. Importantly, we observed a handover between two classes of transcription factors. The early-expressed class has potential roles in lineage biasing, the late-expressed class in lineage commitment. In summary, we provide a comprehensive analysis of lineage commitment at the single cell level, a potential stepping stone to improved lineage control through timing of differentiation cues.