One of the most important problems in development is how epigenetic domains can be first established, and then maintained, within cells. To address this question, we propose a framework which couples 3D chromatin folding dynamics, to a “recolouring” process modelling the writing of epigenetic marks. Because many intra-chromatin interactions are mediated by bridging proteins, we consider a “two-state” model with self-attractive interactions between two epigenetic marks which are alike (either active or inactive). This model displays a first-order-like transition between a swollen, epigenetically disordered, phase, and a compact, epigenetically coherent, chromatin globule. If the self-attraction strength exceeds a threshold, the chromatin dynamics becomes glassy, and the corresponding interaction network freezes. By modifying the epigenetic read-write process according to more biologically-inspired assumptions, our polymer model with recolouring recapitulates the ultrasensitive response of epigenetic switches to perturbations, and accounts for multi-domain conformations, strikingly similar to the topologically-associating-domains observed in eukaryotic chromosomes.