Cells can often choose among several stably heritable phenotypes. Examples are the expressions of genes in eukaryotic cells where long chromosomal regions can adopt persistent and heritable silenced or active states that may be associated with positive feedback in dynamic modification of nucleosomes. We generalize this mechanism in terms of bistability associated with valleys in an epigenetic landscape. A transfer matrix method was used to rigorously follow the system through the disruptive process of cell division. This combined treatment of noisy dynamics both between and during cell division provides an efficient way to calculate the stability of alternative states in a broad range of epigenetic systems.