What is expected utility theory?

It is not the intention here to give a comprehensive account of expected utility theory as good discussions can be found elsewhere.^{14 15} However, it is worth stressing what we mean by “rational” in the context of decision making under uncertainty. An excellent account of rationality is given by Baron¹⁵:

“Rationality concerns the methods of thinking we use, not the conclusions of our thinking. Rational methods are those that are generally best in achieving the thinker's goals. It is true that when we say someone is “irrational” we usually disagree with this person's conclusion, but we disagree in a particular way: we think that better methods ought to have been used in reaching that conclusion.”

Thus, rationality is simply a means of making good decisions—ones which make effective use of the information available at the time of the decision. Of course, in a world of uncertainty, good decisions will not necessarily lead to good outcomes as the best available option will generally include some chance of a bad outcome. For example, patients with atrial fibrillation who are treated with warfarin still have a measurable risk of suffering a thromboembolic stroke. Rather, good decisions may be thought of as those which maximise the likelihood of achieving a good outcome. This is the basis of expected utility theory.

Expected utility theory deals with decisions that can be analysed as gambles. When a simple gamble involves money, the expected value of the gamble can easily be computed by multiplying the probability of winning by the monetary value of the pay off. For example, imagine you were asked to purchase a ticket which allowed you to take part in a game whereby a fair coin is tossed and you win £100 if it comes up heads and nothing if it comes up tails. If the coin was tossed 100 times you would expect that the number of heads and tails coming up would be roughly equivalent (50 each). Thus, you would expect to win £100 approximately 50 times and nothing approximately 50 times, resulting in an average pay off of ((£100 × 50) + (50 × £0))/100 = £50. This is the expected value of the game. Formally:

EV = (p₁ × v₁) + (p₂ × v₂) + (p₃ × v₃)....... + (p_n × v_n)

where EV = expected monetary value, p₁ = the probability of outcome 1, v₁ = the monetary value of outcome 1, and n = the number of possible outcomes.

But would you pay £50 for the ticket which allowed you to play the game only once? When asked this question, individuals normally say that they would pay less than £50 (the expected value) to play this game. Such individuals may be thought of as “risk averse” as the certainty of £50 is worth more to them than a gamble of the same expected value. Nonetheless, the response will vary from person to person. This emphasises the important principle underpinning expected utility theory, that a gamble has a worth or “utility” that cannot be determined by the monetary value alone. To allow for this in assessing the expectation of the game, utility value must be substituted in place of the monetary values used when assessing the expected value. This gives us:

EU = (p₁ × u₁) + (p₂ × u₂) + (p₃ × u₃) ....... + (p_n × u_n)

where EU = expected utility, p₁ = the probability of outcome 1, u₁ = the utility of outcome 1, and n = the number of possible outcomes.

Thus the expected utility theory is the weighted sum of the utilities of all possible outcomes of a course of action, weighted by the probability of that outcome occurring. Since first formalised by Von-Neumann and Morgenstern in 1947,¹⁶ the expected utility theory has been dominant in theories of decision making under uncertainty, largely due to the intuitive appeal of the axioms, or rules, on which it is based. It is unnecessary to describe the axioms in detail, but many are simply the fundamental properties economists usually assume apply to individual preferences. For example, the transitivity axiom states that if A, B, and C are three gambles and A is preferred to B and B is preferred to C, then A will also be preferred to C. Most people would consider this to be a reasonable property for an individual's preferences to exhibit. However, certain of the axioms underpinning expected utility theory are more problematic and violations of the axioms are discussed below.

How does expected utility theory relate to medical decision making?

The decisions facing patients and clinicians may be thought of as gambles since the outcome of any particular course of action is rarely certain, but rather there will exist some probability distribution over a range of possible outcomes. While the probabilities in the examples given above were determined by a simple chance mechanism—that is, the toss of a fair coin—the probabilities attached to all possible outcomes of a medical decision will be determined by the clinical evidence on risks and benefits. Likewise, just as individuals have utility functions over money, they also have utility functions over health states. Utility values for health states are measured on a scale between 0 and 1 (the values usually assigned to death and normal health, respectively) and may be thought of as a measure of the quality of life associated with a particular health state.

Thus, in the context of medical decision making, expected utility theory provides a mechanism for combining information on patient preferences (in the form of utilities) with the clinical information on risks and benefits (in the form of probabilities). We demonstrate below the application of expected utility to one particular clinical decision.

The clinical problem

Atrial fibrillation is common among the elderly¹⁷ and is known to be associated with a greatly increased risk of stroke. Several randomised controlled trials have shown that treatment of selected patients in atrial fibrillation with warfarin significantly reduces this adverse risk of stroke.^18–22 However, treatment with warfarin is known to have adverse effects, most notably the increased risk of suffering a major bleed. In addition, patients' quality of life may be diminished on warfarin therapy by having to undergo frequent blood tests, bruising easily, controlling their lifestyles (for example, physical activity), and controlling their alcohol intake.^{23 24}

Despite the length of time for which evidence of effectiveness of warfarin has been available, and the strength of that evidence, there is still uncertainty over its appropriate use²⁵ with considerable variation in treatment rates and the content of available guidelines.²⁶ Because of the strength of the evidence and the nature of the problem to be addressed, decision analysis seemed an ideal tool to apply to the clinical problem.

Figure 1 shows a highly simplified version of the decision whether or not to treat a patient in atrial fibrillation with warfarin. The decision is denoted by the square node. The chance nodes, denoted by circles, show the outcomes which may occur if that option is followed, along with their probabilities of occurrence. For example, if the decision is “do not treat”, the individual will remain well with a probability of 0.934, suffer a mild stroke with a probability of 0.021, a severe stroke with a probability of 0.015, and a fatal stroke with a probability of 0.03. Note that, in this simplified example, the risk of suffering a stroke is assumed to be two thirds lower in the “treat with warfarin” arm, but with an associated increase in the risk of a stomach bleed from 0 to 0.01.

For the purpose of illustration, hypothetical utility values are attached to each health outcome. Note that the utility of being well on warfarin is lower than the utility of remaining well when not on treatment. The difference denotes the disutility, or loss of quality of life, associated with being on treatment.

Following the method described above, the expected utility of the branch “Treat with warfarin” is:

(0.968 × 0.99) + (0.01 × 0.90) + (0.007 × 0.70) + (0.005 × 0.10) + (0.01 × 0) = 0.97

while the expected utility of the option “Do not treat” is:

(0.934 × 1) + (0 × 0.90) + (0.021 × 0.70) + (0.015 × 0.10) + (0.03 × 0) = 0.95

As the branch “Treat with warfarin” has a higher expected utility than the “Do not treat” branch, the “optimal” decision for a patient whose levels of risk and utility values are identical to those used in this hypothetical example is to treat with warfarin—that is, that by taking warfarin they gain more quality adjusted survival than by not taking the treatment. We can analyse the sensitivity of this result to variability in the probabilities and utility values attached to outcomes by allowing these to vary and re-calculating the expected utility of each branch of the tree. For example, using an utility value for warfarin of 0.96 instead of the 0.99 in the worked example results in a revised estimate of the expected utility in the “Treat with warfarin” arm of 0.94 while that of the “Do not treat” arm remains the same at 0.95, thereby reversing the “optimal” decision. In the same way, attaching probabilities to mild, severe, and fatal strokes in the “Do not treat arm” approximately equal to one third of those in the worked example will also result in a reversal of the optimal decision (continuing to assume that warfarin affords a two thirds reduction in these risks).

Is expected utility theory likely to be valid in clinical practice?

DESCRIPTIVE OR NORMATIVE?

When assessing the validity of a theory, we are usually concerned with testing it either as a description of “what is” or a prescription of “what ought to be”. Thus, a theory may be considered to have descriptive validity if it offers a reasonably accurate account of how individuals actually make decisions under uncertainty. As indicated above, however, violations of the axioms of expected utility theory are not uncommon. For example, the independence axiom states that an individual's preferences over two gambles is independent of irrelevant alternatives. Baron¹⁵ gives an intuitive explanation of the independence axiom along the following lines:

Suppose you were asked to purchase a ticket which allowed you to take part in one of two gambles, the outcomes of which are both determined by the toss of a fair coin. In gamble 1 you win a holiday to the Caribbean if it comes up heads and holiday X if it comes up tails. In gamble 2 you win a holiday to Italy if it comes up heads and holiday X if it comes up tails. The point of the independence axiom is that the destination of holiday X is irrelevant to your preferences over the two gambles: you either prefer a holiday in the Caribbean to one in Italy—in which case you will prefer gamble 1 to gamble 2—or you prefer a holiday in Italy to one in the Caribbean—in which in which case you prefer gamble 2 to gamble 1.

One often discussed violation of the independence axiom known as the “Allais Paradox” is shown in box 1. While the gambles themselves are more complex than in the simple example given above, the basic principle is the same with individuals being asked to choose a preferred gamble out of each of two pairs of gambles. Try it out for yourself. The choices made by the majority of subjects in this situation are inconsistent with the independence axiom and, hence, expected utility.²⁷

As a result, numerous non-expected utility based theories have been developed in an attempt to describe better the way that people make choices under uncertainty.^28–30 For example, in expected utility theory the utility attached to an outcome in one course of action (represented by one branch of a decision tree as shown in fig 1) is independent of the outcomes available under another course of action (represented by another branch of a decision tree). This does not allow for any feelings of “what might have been” had you chosen differently, a consideration which is taken account of in “regret theory”.²⁹ An example for warfarin is given below (adapted from an example given in Ubel and Lowenstein³¹):

“Suppose a physician elicits a utility for severe and mild stroke, a major bleed and being on warfarin. Based on this, the decision analysis suggests she take warfarin to prevent stroke. Unfortunately, she suffers a major bleed and her reaction to this is flavoured by regret: `if only I hadn't taken warfarin'. Having not factored this into the utility assessment, her reaction exceeds the disutility for a major bleed that was predicted by her answers to the standard gamble questions.”

In some respects, non-expected utility based theories such as regret do appear to offer superior descriptive validity in terms of actual behaviour when faced with certain risky choices. However, such models require a more complex representation of individual preferences than is the case with expected utility models and do not offer a comprehensive description of decision making under uncertainty.³²

Another way of looking at expected utility theory is as a normative or prescriptive theory of decision making under uncertainty, as it is based on simple axioms that are held to be rules that any rational person would follow. Baron argues that expected utility theory is normative because it is the decision rule which allows an individual to best achieve his goals in the long run. Provided goals are defined in terms of utility, then this follows logically from the use of expectation—the long run average property of a random variable—in the maximisation process embodied in the equation given above.

However, it has been argued¹⁴ that the long run average outcome only makes sense when considering the average results of many choices. Thus, the long run average must be assessed in one of two ways. Firstly, it may be assessed across the sequential decisions made by one person over a period of time. Cohen argues that most medical decisions are a “one off” as far as the individual patient is concerned, so the concept of the long run outcome of repeated decisions is essentially meaningless. It is worth noting here that our decision model is rather different in that the decision to accept or reject treatment is reversible and subject to annual review, thereby allowing some degree of sequential decision making on the part of the individual.

Secondly, the long run average may be assessed over the choices of many individuals each facing the same treatment decision. For example, if the results of our decision model of warfarin for atrial fibrillation was applied to 100 identical patients, we would expect to maximise the total amount of benefit received across the cohort of patients, the population benefit (although it is likely that certain individuals within this group will have a worse outcome than they would otherwise have done). While it is easy to see how this might provide society with a clear rationale for adopting an expected utility approach, it is less obvious that individual patients or clinicians ought to take into consideration “the greater good” in this way. This raises the prospect that expected utility is normative for society but not for patients and possibly not for clinicians, a consideration which has important implications for the use of expected utility in shared decision making.

While a more thorough discussion of the descriptive and normative properties of expected utility models is beyond the scope of this paper, the crux of the matter would seem to be how the quality of decisions made within expected utility based decision models compares with those made under alternative methods of decision making. We illustrate this point below with reference to our ongoing research.

HOW COULD WE APPLY A DECISION ANALYTICAL MODEL IN PRACTICE?

We have taken an expected utility based approach in a project which set out to construct clinical guidelines for the use of anticoagulants in patients with atrial fibrillation using decision analysis.³³ Though not shown here in detail, our results demonstrated that the optimal decision for any one patient is sensitive to a number of key variables in the model, particularly the disutility of being on warfarin therapy—that is, the effect of warfarin treatment per se on quality of life. Furthermore, there was much interpatient variation both in the valuation of health states and in the magnitude of the risk of stroke. For example, the interquartile range of the utility estimate of being on warfarin therapy was found to be 0.92–1.00 (no disutility) with a median of 0.986, while that for mild stroke was found to be 0.48–0.83 with a median of 0.675. Thus, the optimal decision for an individual patient may be very different from that generated by a model which makes use of population-average values or, indeed, are suggested by a clinical advisor.

To illustrate this general point we have applied the results of our decision analysis model to a cohort of patients in atrial fibrillation and found that between 22% and 96% of men over 75 would benefit from treatment depending on whether the lower or upper interquartile value for the utility of warfarin was used. Thus, because of the variability in patients' views on the impact of warfarin therapy on having a stroke and in their own individual risk of stroke, it is difficult to provide straightforward advice on warfarin treatment without having the means to assess both. Furthermore, clinicians' preferences are likely to differ from patients' preferences in the same way that patients' views differ from each other.

These problems encountered in attempting to apply population based approaches to individual patients, or in attempting to advise patients on treatment choice without fully understanding their values or preferences, highlight the need for research into how decision analysis may be used at the level of the individual patient consultation—the DARTS project (see Appendix).

BETTER DECISION MAKING

As argued above, we consider the real test of the benefits of models such as ours is that they improve the quality of the shared decisions reached. Although “better” decision making is an ill defined concept, it is generally accepted that good decisions are those which are “well informed”. In conducting a systematic review of informed decision making, Bekker et al¹ have discussed the problems associated with defining “informed” and offer the following:

“An informed decision is one where a reasoned choice is made by a reasonable individual, using relevant information about the advantages and disadvantages of all possible courses of action, in accord with the individual's beliefs.”

Alternatively, O'Connor² has defined an effective decision as “informed, consistent with personal values and acted upon”. So, in arriving at an informed decision the patient must not only have the necessary information but also be able to process this information in a way which truly reflects their preferences. But individuals are known to have limited information processing capabilities and to adopt simplifying strategies when asked complex evaluative tasks.³⁴³⁵ The use of such simplifying strategies may result in suboptimal decisions being made. For example, patients may focus too closely on one particular piece of information, say the outcome of severe stroke, and fail to take account of the (possibly very small) likelihood of this occurring or of the likely impact of warfarin on their quality of life.

Such findings, along with the growing desire for shared decision making, has resulted in the increasing use of decision aids to support the patient in making clinical decisions.³⁶ For example, the work of Wennberg has led to the development of interactive computer programmes and video disks for patients which enable them to make better informed decisions based upon a presentation of the clinical evidence and likely effects of alternative approaches. This has been most extensively developed and evaluated in the case of men with prostate symptoms who seek to choose between prostatectomy and “watchful waiting”.³⁷ Other examples include decision boards developed to help women make decisions surrounding breast cancer treatments³⁸ and audiotapes and booklets which address a range of clinical decisions including warfarin for atrial fibrillation.²

Unfortunately, there is no gold standard for optimal decision making involving patient values.³¹ Just as patients' values are known to be variable, so too is their desire for information and for participation in the decision making process. Some patients actively seek and desire information and active involvement in the decision making process while others prefer to devolve clinical decision making to their clinical advisor (so called “monitors” and “blunters”).³⁹ Furthermore, patient preferences on their information needs can influence their responses to its provision, such that “blunters” given information may suffer more anxiety in decision making than if the information was absent.⁴⁰ Similar variation in information needs has been shown in patients with breast cancer.⁴¹

In the light of this, proponents of better informed patient decision making based upon decision theory suggest two possible ways forward.⁴² Firstly, an individual's values may be assessed explicitly, such as in the standard gamble exercise discussed above, and “plugged into” the type of decision analysis model shown in fig 1, along with estimates of clinical risks and treatment effectiveness, and an optimal decision identified which is specific to that patient. This reduces the cognitive burden on the patient to a minimum as they are not required to combine and integrate all of this information; rather, it is done on their behalf.

The DARTS project has been developing a tool to do this by means of a three stage process. The prototype consists of three components: the derivation of patients' values for the relevant health states (for example, major/minor stroke, warfarin treatment, major bleed) using a standard gamble method; presentation of risk information for an individual patient using the Framingham stroke risk equation in conjunction with estimates of the effectiveness of warfarin at reducing this risk and the associated risk of a major bleed; and generating an “optimal” decision for the individual patient by combining these data in a Markov decision analysis model. Further details of the decision analysis model and the clinical evidence used within it have been published elsewhere.³³

A second proposed approach suggests that decision models may be used to support patients in incorporating their values without formal value elicitation methods. While Dowie is a proponent of the explicit approach, his proposal for the use of clinical guidance trees which incorporate discussions of risk and values into the clinical interaction, supported by computerised decision trees, offers a systematic approach that does not necessarily require the derivation of individual utilities.⁴³ This approach would appear to involve a greater cognitive burden on both the patient and practitioner but also allows for the inclusion of more intuitive factors absent from conventional decision analysis models.

This work is still in the developmental stage and there is no evidence to determine how the quality of decisions made in this manner compare with those generated by conventional decision models, by use of alternative decision aids such as decision boards, or with those taken within an altogether different model of medical decision making. In the DARTS model this latter option could be applied by simply taking the patients through the risk presentation component of the tool, asking them to consider their preferences over the various health outcomes (but not eliciting their values explicitly in a standard gamble exercise), and then inviting them to use this information in weighing up the advantages and disadvantages of treatment.

EVALUATION

There remain considerable challenges in evaluating the effectiveness of decision analysis models (or other decision aids). This has been discussed by Entwhistle et al⁴⁴ and depends to an extent upon the intended outcome of the decision aid. In our case we have developed a tool that is intended to promote shared decision making—that is, not one that necessarily leads to patients making decisions consistent with expected utility theory. Nor, however, can we assume that decisions made in this way will necessarily lead to fewer strokes or side effects. Rather, the model assumes that patients who are better informed and engaged will make “better” decisions and that appropriate measures of effectiveness are likely to include greater satisfaction with the decision, less anxiety, and better informed patients with greater concordance with treatment.

While further research is required into what makes a better decision, some progress has been made towards developing tools which make use of proxy measures such as patient satisfaction⁴⁵ and lack of decision conflict.⁴⁶ For example, the decision conflict scale is a 16 item instrument which sets out to explore (a) the patient's uncertainty about choosing among alternatives, (b) the factors contributing to the uncertainty, and (c) perceived effective decision making. Scores on each of these three subscales are summed and the total score is taken to be a measure of the patient's decision conflict surrounding the decision.

Another proxy for a “better” decision may be one with which the patient is more likely to comply. Failure to take prescribed medication may limit the effectiveness of treatment and represents a waste of resources. One view of poor compliance is that the treatment decision has failed either properly to inform the patient and/or to consider the preferences: there is evidence to suggest that better informed patients are more likely to be compliant. Indeed, the term concordance has recently been suggested to replace compliance, reflecting changing views on patients' reasons for not taking or completing courses of drugs.⁴⁷ Thus, an approach that engages patients in better understanding their values and the risks and benefits of treatment may also help with their commitment to the treatment.

Conclusion

Expected utility based decision analysis models provide a potential mechanism for integrating patient preferences with probabilistic information in a systematic and explicit way. The use of decision analysis models in clinical decision making might be expected to reduce the cognitive burden on the patient, an important factor in light of the evidence that individuals are known to make systematic mistakes when dealing with probabilistic information.³⁵ Furthermore, in making the decision making process explicit, the use of decision analysis allows the uncertainty surrounding a decision to be quantified and allows us to look at the implication of small changes in any of the key variables.

On the other hand, conventional decision analysis models of the type described above will inevitably fail to capture all the considerations which may influence patient and clinician choice. It would not be surprising therefore if the actual decisions made by doctors and patients on occasions differ from the “optimal” decision generated by the model. Such a finding would not in itself render decision analysis models redundant in aiding clinical decision making. Rather, we have argued that the usefulness of decision analysis models has to be assessed with reference to their ability to aid “better” decision making and this need not require the actual choices to conform with the axioms of expected utility theory. The real benefit of using decision analysis models in clinical decision making may lie in their ability to help both patients and clinicians to structure the decision into a manageable form, to consider the likelihood of each outcome occurring along with their preferences over those outcomes, and to combine these in a way which reflects their beliefs. There may also be accompanying educational benefits if, for example, decision analysis models make evidence based information more readily accessible to clinicians and patients in a useable form.

Thus, if adopting a decision analytical approach results in less decision conflict, more patient and clinician satisfaction, greater compliance with prescribed medication, and improved health outcomes, then clearly decision models have a role to play in improving clinical decision making. The results of studies such as ours will begin to shed light on this issue which has important implications for the quality of clinical decision making.

Appendix: The DARTS study

PURPOSE

The purpose of the study was to develop our recently published decision analysis of warfarin for atrial fibrillation into a computerised tool to aid shared decision making between clinician and patient, and to assess the acceptability and feasibility of such a tool for use in routine clinical consultations.

METHODS

We iteratively developed a desktop software program using interviews and focus groups with clinicians and patients and carried out a training and testing phase using volunteer clinicians and “dummy” patients. We then carried out a small scale feasibility study in which those clinicians trained in the use of the tool used it in consultations with “real” patients. Data collection methods used included audio and video recording, observations of consultations, debriefing interviews, and patient questionnaires.

RESULTS

The key results from the work so far have shown that it is feasible to take older patients through a decision analysis based computerised tool, including a standard gamble procedure; patients and clinicians welcome access to information on risks of stroke and major bleed both on and off treatment; care must be taken to avoid the tool appearing too “directive” either to patients or clinicians; the tool requires time and training to use, particularly the standard gamble component. The prototype consists of three components: (1) derivation of patient values for relevant health states—for example, major/minor stroke, warfarin treatment, major bleed—using a standard gamble method; (2) presentation of the individual risk of stroke for the patient from their risk profile using the Framingham equation and the benefits/risks of warfarin therapy from literature review; and (3) agreement of a shared decision with reference to the output from a Markov decision analysis model which uses these data.

CONCLUSIONS

The results of our study to date have shown that it is feasible iteratively to develop a decision analysis based computer software package that is acceptable to elderly patients and clinicians alike. We intend next to undertake an intervention study of the computerised tool.