When constructing decision-analytic models to evaluate the cost-effectiveness of alternative treatments, we often need to extrapolate beyond the available experimental data, as these typically relate to a limited period starting from the initiation of a new treatment or the diagnosis of the current disease state. We may also be required to extrapolate beyond the available experimental evidence to compare potential treatment sequences. Markov models are often used for this extrapolation. These models have the defining assumption that future transition probabilities are independent of past transitions. This means that, in general, transition probabilities cannot be conditional of the time spent in a given state. Where data exist to show that the risks of transition are conditional on the time spent in the treatment state, the simplifying Markov assumption can result in a loss in the model's "face validity," and misleading results might be generated. Several methods are available to incorporate time dependency into transition probabilities based on standard methods and software. These include the inclusion of tunnel states in Markov models and patient-level simulation, where a series of individual patients are simulated. This article considers the features and limitations of these methods and also describes a novel approach to building time dependency into a Markov model by incorporating an additional time dimension resulting in a "semi-Markov" model. An example of the implementation of such a model, using the R statistical programming language, is illustrated using a cost-effectiveness model for new epilepsy therapies.