Modelling in economic evaluations of medicines

Economic evaluation in health (also known as pharmacoeconomic in case of medicines) identifies, measures, and values costs and outcomes of alternative healthcare technologies, and can be performed alongside controlled clinical trials, but analytical modelling is usually used. Decision trees and Markov models are the two most common approaches used in economic evaluation. The biggest advantages of a decision tree are clarity, simplicity, and straightforwardness. On the other hand, the main advantage of the Markov model is its ability to incorporate complex events into the simulation, which is practically impossible to do with a decision tree. Reimbursement policy in Serbia mandatorily incorporates economic evaluations to promote availability and accessibility of the prescription medicines. To show current pharmacoeconomic value of a medicine, budget impact analysis and the cost-effectiveness analysis should be included. The latter should be conducted using appropriate modelling techniques. However, since no official methodological guidelines about the modelling and economic analysis exist, the submissions by marketing authorization holders vary greatly. The future of pharmacoeconomic modelling depends on the research area of interest, with new frameworks and approaches being developed.


Introduction
Modern health systems are under enormous financial pressure. In this context, it is increasingly important to demonstrate that new healthcare technologies provide consistent health gains at reasonable costs compared to alternative use of resources.
The methodology commonly used to analyse this issue is called economic evaluation. Economic evaluation can be defined as a set of research methods that identifies, measures, and values costs and outcomes of alternative healthcare technologies (1). When economic evaluation is applied specifically to pharmaceutical products, it is known as pharmacoeconomics. Economic evaluation in healthcare has its foundation in economic science, especially welfare economy, but also applies a wide range of scientific disciplines such as epidemiology, psychology, or decision analysis (2,3).
Economic evaluations are currently used in two main contexts: (I) in decision making to optimise medicine prescribing by health care professionals, and (II) in decision making by regulatory authorities and policy makers in respect to reimbursement of medicines.
Economic evaluations can be performed alongside controlled clinical trials (4), but usually, analytical modelling as an alternative option is used. Modelling in economic evaluation implies the use of mathematical techniques and computer software to incorporate information from multiple different sources to evaluate costs and outcomes of alternative healthcare technologies. Various models can be used in the economic evaluation of healthcare (5, 6), but decision trees and Markov models remain the two most common approaches (7,8).

Decision trees
A decision tree is the simplest analytical model used in economic evaluation. Figure  1 shows a hypothetical example of a decision about the treatment of a disease X. There are two possible alternatives, represented as branches: no therapy or start therapy with medicine A. The choice between these two alternatives is represented graphically as a square or decision node. Independently of the chosen alternative, the clinical pathway allows the patient's recovery or its alternative without recovery, which is characterised by chance node (circle). All events in the clinical pathway that follow each option are linked with probabilities and must be mutually exclusive. The sum of probabilities for each alternative must be the value of one. At the end of each branch is a terminal node, represented by a triangle, to which consequences of health technology, called payoffs, are assigned. The payoffs can be any type of outcomes: clinical (e.g. life years), economic (costs), or humanistic (e.g. quality-adjusted life years, QALYs). QALY, as a single measure, combines the patient's quantity (life expectancy) and quality of life, thereby providing a common index that facilitates comparisons of therapy outcomes across different diseases. QALY is calculated by multiplying the time spent in a certain health state by the utility or preference weight linked with that state. The utility weights can be obtained by direct or indirect methods (9). Hipotetički izgled modela drvo odlučivanja The probabilities (expressed in form of rates, counts, relative risks or odds ratios) and payoff values can usually be found in published literature, but in the absence of publicly available data, expert opinions can be used. The costs can be calculated using official price lists for medicines or health services.
Once the decision tree has been properly designed and all probabilities and payoffs added, the evaluation of the model can be performed. This process is also known as "rollback" since the calculation is done by multiplying the values of payoff (right side of the tree) with the respective values of probability (left side of the tree) and summing up all these values (10,11).
The greatest advantages of a decision tree are clarity, simplicity, and straightforwardness. The main disadvantage of a decision tree is the omission of its use in time-dependent models, such as in chronical diseases where the events are recurring and where transition of health events occurs.
The application of decision trees in clinical practice in Serbia has been recognized in different areas, including screening procedures (12), pharmacotherapy of infectious diseases (13,14), labour for singleton pregnancy (15) and pharmacist service (16).

Markov models
Markov models (named after the Russian mathematician Andrey Andreyevich Markov) include recurring clinical events that happen during a certain period of time. As such, they are applied in economic evaluations of medications used in different chronic diseases or conditions. Any Markov model consists of a finite number of mutually exclusive health states. At any point in time, the patient can reside in only one of those health states. The movement of the patient between health states is defined by transition probabilities. In the Markov model, time is presented in discrete intervals called cycles, at which the patient can move between health states or remain in the current one (1,11).  In the Markov model, the sum of transition probabilities out of health states must be one. Let's say that the probability of moving from "well" to "sick" health state is 0.2 at the end of each cycle. At the same time, probability of moving from "well" to "dead" health state is 0.1. This means that the probability of remaining in health state "well" must be 0.7 (calculation: 1-0.2-0.1=0.7). The process of transition is repeated after each cycle until a termination rule is reached. Usually, the termination rule is defined by the specific number of cycles, or until the entire cohort (population) reaches the so-called absorbing state, meaning the state that can't be left. In Figure 2, this state is "dead".
As previously stated, values for probabilities and payoffs can be found in and taken over from published literature or be defined by medical experts. The evaluation of the Markov model is like the decision tree. Payoffs are assigned at each health state.
Payoffs can be expressed as costs, life years, utilities, etc. The expected values are calculated in a similar manner, by multiplying the percent of the cohort in each state with the costs or outcomes assigned to that state. Finally, the expected costs and outcomes are totalled across the cycles.
The main advantage of the Markov model is its ability to incorporate complex events into the simulation. This is practically impossible to do with a decision tree. Moreover, one of its key strengths is the ability to translate data from the individual level to the level of population. On the other hand, this advantage may potentially lead to one of the model's limitations, such as disregard of heterogeneity (17). Microsimulation models and their extension, agent-based modelling, are proposed to overcome the limitation on heterogeneity when modelling the distribution of individual lifespans is needed (knowing the individual progress over time) (17,18).
However, the main limitation of the Markov model is memorylessness or Markov assumption, which means that the probability of a given transition in the model is independent of the nature or timing of earlier transitions. In the given example, this corresponds to the probability of dying being independent of the time the individual spent in the "well" health state before moving to the "sick" health state.

Economic evaluations and modelling in Serbia
In an attempt to control expenses and evaluate the value of new medicines, economic evaluations are increasingly used by decision makers, regulatory bodies, and health insurance funds all around the world.
Besides the procedure for obtaining marketing authorization for a medicine at the Medicines and Medical Devices Agency of Serbia, the marketing authorization holder needs to fulfil additional steps to ensure the availability of the medicines in the market. The first is to determine the maximum wholesale price for the medicine. The institutions responsible for this step are the Ministry of Health and Ministry of Trade, Tourism and Telecommunications. According to the regulation, one of the mandatory criteria in determining the maximum medicine price is pharmacoeconomic evaluation (25). The term "pharmacoeconomic evaluation" is used only to mark selected indicators, such as cost per daily defined dose, monthly or annual costs of therapy, and comparative analysis of costs and outcomes of medicine use (25). Although the last indicator corresponds to the definition of economic evaluation (1), current practices show that different types of submitted materials are accepted. At this point, marketing authorization holders may prepare and submit one of the models that describe and evaluate costs and outcomes of the medicine. However, this is usually not done.
After obtaining the maximum price of medicines, the marketing authorization holders may distribute medicines through pharmacies. However, for a significant majority of prescription medicines, the final step is the application for complete or partial reimbursement of medicines by the Republic Fund of Health Insurance. In this way, the marketing authorization holder assures better availability of the medicines, which is especially important in the case of new, innovative medicines.
The Republic Health Insurance Fund is the authority responsible for making the decision on the reimbursement policy of the medicine. According to the bylaw that regulates conditions, criteria, way, and procedure for reimbursement of medicines, the request for medicines to be included in the Positive list should comprise, among other documents, the results of the pharmacoeconomic evaluation of medicine (26). Among the pharmacoeconomic evaluations, analyses that are required are: budget impact analysis and cost-effectiveness analysis (for: innovative medicines, new original medicines and where a comparative medicine exists and is already included within the Positive list), that "should be conducted using appropriate modelling techniques (decision tree, Markov model, Monte Carlo simulation -multiple simulations of a model often used in uncertainty analysis, or other similar models), customized according to the current situation in the Republic of Serbia" (26). At this point, the marketing authorization holder is obliged to prepare and submit one of the models that describe and evaluate costs and outcomes of alternative medicines. However, since no official methodological guidelines about the modelling and the economic analysis in general exist, the submissions vary greatly. The existence of guidelines would increase consistency, quality, and uniformity of the submitted files, but also the transparency of the whole process. The development of the guidelines is usually done in two stages, in which experts from academia play an important role (27,28).

Modelling in public health interventions
Development and promotion of different programs for raising awareness about the importance and significance of health promotion, prevention of different diseases and their complications have been rapidly increasing in the past years (29).
Along with the two main contexts of economic evaluation (optimising medicine prescribing by health care professionals, and support in decision making by regulatory authorities and policy makers in respect to the reimbursement of medicines.), its application can be also widened with the inclusion of public health interventions (PHIs) by health care professionals. Two important questions need to be answered before implementing the new PHI: "Is the new PHI worthwhile?', and "Could the money for the new PHI be better spent on alternative programme(s)?". Economic evaluation and modelling can help with both questions (29).
It is considered that traditional economic evaluation methods are well established, but insufficient for the assessment of PHI. With respect to that, economic modelling can be an ideal tool for PHI assessment, even though some challenges exist. It has been proposed that the decision analytic model (DAM) can be used for this purpose (29). This is a mathematical decision-making tool used to integrate evidence on costs and outcomes from different sources in order to notify decision makers about the resource allocation in public health services and clinical practise (29).
Briggs and colleagues (11) suggested and adopted decision analysis as a robust framework when conducting economic evaluations of health technologies and PHIs, and they propose basic steps for DAM provided (Figure 3).

Figure 3. Framework for conducting the decision analytic model Slika 3. Okvir u sprovođenju analitičkog modela odluke
The third step of the DAM requires the application of pharmacoeconomic analyses. The most commonly used analyses are decision tree, Markov model, discrete event simulation, agent-based modelling, and system dynamics. Dynamic models are considered to be the proper model for modelling infectious diseases, and it is supposed that they also can be used for modelling in PHIs. In those cases, the DAM model should be determined by exploring different factors such as decision-makers' requirements, the complexity of the disease or health care area, modellers' expertise, or preference (29,30).
There are many challenges for modelling in PHIs. Squires and Boyd summarized the features of public health systems along with modelling challenges, current practices, and latest methodological research that could provide potential solutions for challenges in practice (29). Approaches suggested to try and overcome challenges are early-stage decision modelling with an iterative approach to economic evaluation of PHIs. The stages in iterative approach are provided in Figure 4 (29,31). According to Squires and Boyd, the key principles of good practice for developing the structure of public health economic models are: (I) system approaches to public health modelling should be taken into account; (II) understanding of the public health problem is an imperative prior to and alongside developing and justifying the model structure; (III) strong communication with stakeholders and members included in the model development is essential; (IV) a systematic consideration of the determinants of health is central to identifying all key impacts of the interventions within public health economic modelling (29).

Future of modelling
The decision tree and Markov model are still the most widely used analytical modelling techniques in health evaluations. However, the future of pharmacoeconomic modelling depends on the area of research interest and demand for modelling. New frameworks and approaches, as well as new modelling techniques are being developed and future research should pay more attention to widening and promoting their usage (e.g., conceptual modelling framework, early-stage decision modelling, iterative framework). These new approaches could overcome challenges that are present in current practice and usually used models. Since the interest in public health has rapidly increased in the last decade, this area could benefit the most from modelling in economic evaluations, and consequently also society, healthcare systems, decision makers and any other stakeholders.