Objective: Cancer stem cells (CSCs) have been hypothesized to initiate and drive tumor growth and recurrence due to their self-renewal ability. If correct, this hypothesis implies that successful therapy must focus primarily on eradication of this CSC fraction. However, recent evidence suggests stemness is niche dependent and may represent one of many phenotypic states that can be accessed by many cancer genotypes when presented with specific environmental cues. A better understanding of the relationship of stemness to niche-related phenotypic plasticity could lead to alternative treatment strategies. Methods: Here we investigate the role of environmental context in the expression of stem-like cell properties through in-silico simulation of ductal carcinoma. We develop a two-dimensional hybrid discrete-continuum cellular automata model to describe the single cell scale dynamics of multi-cellular tissue formation. Through a suite of simulations we investigate interactions between a phenotypically heterogeneous cancer cell population and a dynamic environment. Results: We generate homeostatic ductal structures that consist of a mixture of stem and differentiated cells governed by both intracellular and environmental dynamics. We demonstrate that a wide spectrum of tumor-like histologies can result from these structures by varying microenvironmental parameters. Conclusion: Niche driven phenotypic plasticity offers a simple first-principle explanation for the diverse ductal structures observed in histological sections from breast cancer. Significance: Conventional models of carcinogenesis largely focus on mutational events. We demonstrate that variations in the environmental niche can produce intraductal cancers independent of genetic changes in the resident cells. Therapies targeting the microenvironmental niche, may offer an alternative cancer prevention strategy.