Recent advances in single cell transcriptional profiling open up a new avenue in studying the functional role of cell-to-cell variability in physiological processes such as stem cell differentiation. In this work, we developed a novel algorithm called SINCERITIES (SINgle CEll Regularized Inference using TIme-stamped Expression profileS), for the inference of gene regulatory networks (GRNs) from single cell transcriptional expression data. In particular, we focused on time-stamped cross-sectional expression data, a common type of dataset generated from transcriptional profiling of single cells collected at multiple time points after cell stimulation. SINCERITIES recovers the regulatory (causal) relationships among genes by employing regularized linear regression, particularly ridge regression, using temporal changes in the distributions of gene expressions. Meanwhile, the modes of the gene regulations (activation and repression) come from partial correlation analyses between pairs of genes. We demonstrated the efficacy of SINCERITIES in inferring GRNs using simulated time-stamped in silico single cell expression data and single transcriptional profiling of THP-1 monocytic human leukemia cell differentiation. The case studies showed that SINCERITIES could provide accurate GRN predictions, significantly better than other GRN inference algorithms such as TSNI, GENIE3 and JUMP3. Meanwhile, SINCERITIES has a low computational complexity and is amenable to problems of extremely large dimensionality.