Human gut microbiota is a very complex and dynamic ecosystem that plays a crucial role in our health and well-being. Inferring microbial community structure and dynamics directly from time-resolved metagenomics data is key to understanding the community ecology and predicting its temporal behavior. Many methods have been proposed to perform the inference. Yet, we point out that there are several pitfalls along the way, from uninformative temporal measurements to the compositional nature of the relative abundance data, focusing on highly abundant species by ignoring or grouping low-abundance species, and implicit assumptions in various regularization methods. These issues have to be seriously considered in ecological modeling of human gut microbiota.