Societal Preferences for Healthcare

Research into how to allocate our scarce healthcare resources

The first published paper from my PhD

This feels like a milestone: the first paper from my PhD research has been published. The paper is based on the pilot survey, which was intended to identify the preferred method for eliciting societal preferences, and compares discrete choice experiments (DCE), which are relatively common in health economics, with constant-sum paired comparisons (CSPC), which are relatively less common. Therefore, this paper discusses the strengths and weaknesses of the two methods with respect to eliciting societal preferences, rather than societal preferences per se.

Briefly, the two methods appear to invoke different response behaviours among respondents.  DCEs ask respondents to select a preferred option from a pair of alternatives, and the observed choices appeared to reflect a competitive approach to the tasks as respondents were slightly more likely to focus on differences in one or a few key attributes in selecting a ‘winner’.  CSPC tasks, in contrast, ask respondents to allocate budget percentages between the two alternatives.  This difference in response format would seem to require a more reflective approach to the task, as respondents must, in effect, judge the value of one alternative relative to the other and is argued to encourage a more holistic consideration of the alternatives.

As respondents to the CSPC questionnaires moved a slider to allocate the budget between alternatives, the number of patients that could treated, and the aggregate quality-adjusted life years (QALYs) gained, moved in proportion to the budget allocated to that alternative.  It was expected that seeing these numbers move as they moved the budget slider would give more weight to these attributes, known as a ‘prominence effect‘,  which suggests that respondents become more sensitive to an attribute when it is harder for them to ignore.  The results appeared to be consistent with this hypothesis, as the number of patients treated was the third most important attribute in the CSPC results, but was statistically insignificant in the DCE.  However, respondents to the CSPC were no more likely to choose the alternative associated with the greatest aggregate QALY gains than respondents to the DCE.

Ultimately, there was little to choose between the two methods.  However, as the allocation of healthcare resources often involves difficult decisions over important rights and principles that people may be reluctant to trade-off, it was felt that CSPC tasks — by allowing respondents to allocate some share of the budget to their less preferred alternative — may have had an advantage over the extreme, ‘all-or-nothing’ decisions of the DCE.  Given the interesting contrasts and differences observed in the relatively small pilot study, it was decided to use both methods in the larger primary survey of preferences.

If you are interested in reading the published article but can’t access it through the publisher’s website above, an earlier draft of the manuscript is available here.  I will also be presenting results from the DCE portion of the primary survey at a conference at the end of June, and I hope to post some of those results here when I return, so please keep an eye out for those.


A distraction from real work: alternatives to Google Reader

Google announced today that they would be retiring Google Reader, the application I prefer to use for following blogs online.  This was quite disappointing as I am a fan of Reader’s straightforward interface.

If you are using Reader to follow this blog or any other — and there is a good chance you are, since it is by far the dominant RSS reader out there — and you want to continue following those blogs using a different reader, your first step should be to export your list of subscriptions.  Fortunately Google has made this relatively painless using their Takeaway service, which lets you export your data from different Google products.  Follow this link to export your Reader subscriptions.  Click on Create archive, which will create a .zip file with your subscriptions as well as a list of specific starred and shared posts, and then download that file to your computer.  Unzip it and keep particular track of the file called subscriptions.xml.  You can upload this file to most other reader applications to have instant access to your subscribed blogs.

I’ve tried a few alternatives to Google Reader so far, and in keeping with the theme of this blog, they all have tradeoffs.  My preferred alternative was, an online reader that is the most similar to Google Reader of all the options I tried.  Setting up an account was a little more difficult than some of the readers, which let me sign in using my Google usename, but overall it was still relatively simple.  Importing my subscriptions.xml file was easy enough through the Add content tab at the top left, although not as easy as some serives that allowed me to sign in using my Google username and password.

The default appearance makes it look like a web portal rather than a reader, which I found a little off-putting, but you can easily switch between what they call the widgets view and the reader view I was more comfortable with.  I also had to tinker with the settings to get it to display the full posts as the default rather than just tiled snippets of different posts, which I’m learning, is the preferred look for most of the different readers I tried.  Once I got the settings sorted, I’m quite happy with Bloglines.  The layout is similar to Reader, with a list of subscriptions at the left, and the posts to the right.  I can scroll through each blog and it automatically marks the posts as read as I scroll past them.

Update: Since starting to use Bloglines I have noticed that it is very slow to refresh.  Whereas Google Reader refreshes more or less instantaneously, I’ve noticed the lag between a post being published and showing up in Bloglines has been about 8(!) hours.  I am hoping that this is just a symptom of the massive influx of new users they have experienced in the last two days, but there are forum posts to suggest that this is an ongoing issue dating back 5 years or more.  This will be a dealbreaker for me if it continues for more than the next few days.

The other main contender in my mind was  This is a browser add-on rather than a true online reader, but it works with Firefox and Chrome, and iPad.  With its magazine style layout, Feedly seems to be aimed more at the iPad user than the desktop user.  It is certainly prettier than Bloglines, but I found it difficult to re-create the simple Reader interface I was looking for.  Although it’s possible, there is a lot of whitespace that may look good on an iPad, but I found it annoying on a PC.   Once I installed the Firefox add-on from the feedly homepage, getting started was a snap as I could sign in using my Google username and Feedly automatically imported my subscriptions.  Indeed, at the moment Feedly and Reader happily co-exist, with changes in one instantly showing up in the other.  Obviously this will become moot once Reader is retired, but it helps ease the transition.  As you might expect, there are a multitude of themes to choose from, but again, they were largely lost on me.  The key drawback to feedly in my view was the fact it was an add-on rather than a true online reader and would have to be installed on any device you wanted to use.  Obviously this is a minimal hassle for most people, but together with my personal aversion to the magazine-style layout, it was enough to convince me to go with Bloglines.

As you search for your own preferred alternative to Reader, you may also find this more detailed review of different RSS readers helpful, as I did.  Also, you may also want to take the advice here: wait until Google Reader shuts down on July 1st before looking for a replacement.  Now the dominant player is exiting the market, new players will be entering and there may be options by July that we can’t even anticipate today.

Final update: It’s the week before Google Reader shuts down for good, and although I haven’t seen any announcements of a major new player among the RSS readers, I have just found an online reader that I am very happy with.  I had settled into a ‘good enough’ relationship with Bloglines, but just recently came across CommaFeed.  It’s free and it’s open source, both of which are tremendously appealing to me, but most importantly, it is a very good reader.  It has a nice clean interface, very reminiscent of Google Reader, and it is much, much faster than Bloglines to update feeds.  It also offers an option to hide subscriptions with no new posts, reducing clutter and making it easy to spot unread articles.  This had been a minor but nagging irritation with Bloglines.  Importing your subscriptions from Google Reader is as simple as allowing CommaFeed one-time access to your Google Reader settings (don’t worry, you’re not giving out your Google password).  Feeds are imported and populated almost instantly.  There are limited style options (you can choose between displaying just titles or the full posts) and you can set the interface to any colours as long as they are black and white, but overall the layout is simple and appealing.  I think I’ve found my winner!

I realize this post has nothing to do with societal preferences, and I apologize for the lack of activity lately.  Most of my time the last few months has been spent writing as I work toward a full draft of my thesis by this spring.  I have completed a number of interesting analyses, though, and I hope to post those results here soon.

Upcoming panel discussion, Oct 19

I will be participating in an upcoming panel discussion of patient-level priority setting in healthcare along with Paula Adam of the Catalan Agency for Health Information, Assessment and Quality.  We will be discussing the rationale for individual-level priority setting, different methodological approaches, and the results of two patient-level priority setting exercises in European and Canadian contexts.  For those in Halifax, the details of the presentation are as follows:

Who should have priority in access to health care? European and Canadian perspectives
Friday 19 October 2012, 3:30-5:00pm
Dalhousie University, Mona Campbell Building, Room 1108
1459 LeMarchant St, Halifax N.S.  (corner of LeMarchant and Coburg)

I hope you are able to attend what should be an interesting and provocative discussion.  For those unable to attend, I will post a copy of my slides following the presentation.

Decision-making in healthcare

I recently helped conduct a workshop on Measuring Societal Preferences along with Dean Regier and Helen McTaggert-Cowen at the Priorities in Health 2012 conference in Vancouver, BC.  The workshop covered why we might be interested in measuring societal preferences, qualitative approaches to eliciting preferences, and quantitative approaches, including experimental design for choice experiments and a brief overview of stated preference methods.

My contribution to the workshop (my slides are here) was a discussion of the theoretical underpinnings for measuring societal preferences in healthcare, and specifically why we might want to try to measure people’s preferences.  Welfare economics has conventionally taken a “welfarist” perspective, which holds that individuals are the best judges of their own well-being, and that each individual’s well-being is as important as everyone else’s.  It is therefore inappropriate for anyone, and most particularly ‘society’, to try make decisions on behalf of others.  Under this perspective, the only fair way that we as a society can decided whether or not an option is worth doing (for example, should the government buy a new piece of medical equipment?), is to check whether each individual citizen feels better off with the new option.  If everyone says yes, then we should buy the new equipment.  If somebody — even just a single person — says they feel worse off, then we should not buy the new equipment.  Because one person’s well-being is just as important as everyone else’s, we cannot disregard anyone’s well-being for the sake of “the greater good.”  To do so would be to prioritize one person’s well-being over another’s.

However, because most societal decisions involve ‘winners’ and ‘losers’ — in our example, everyone might be taxed in order to buy a piece of equipment that only a few of them may ever need — a strict welfarist perspective is somewhat impractical.  In addition, the idea that resources cannot be redistributed unless everyone agrees means that blatantly unfair allocations, for example, where all resources in society are held by a single individual and everyone else has nothing, can be held as ‘fair’ if that single person objects to a redistribution.  For this reason, the strictly unanimous decision rule has been modified to allow for the possibility of the winners compensating the losers.  If the winners from a particular decision could, in theory, compensate the losers and still remain better off than they were before the decision, the new program has an overall net benefit to society and we should proceed.  Welfare economics takes the position that whether or not compensation actually takes place is a political, not economic, issue, and all that matters from an economic perspective is that there is the POTENTIAL for compensation that leaves everyone at least as well-off as before the decision.  Also recognize that now we’re allowing for some people’s preferences — the winners — to over-ride other people’s preferences — the losers.  This combination of over-riding some people’s preferences, while also allowing for the possibility of winners compensating losers, is known as the “extra-welfarist” perspective (‘extra’ in the sense of “factors beyond just individual welfare”).

This decision rule works well in most cases, and ensures that decisions that would have a net benefit to society can proceed on the grounds that decisions can be made for the greater good, while losers can be compensated for their losses.  Things are more complicated, though, when we consider healthcare decisions.  A new piece of healthcare equipment may help a person live a longer life, or have a greater quality-of-life.  Such benefits are difficult to value in terms of dollars and cents, and therefore are more often measured in terms of quality-adjusted life years, or QALYs, but maximizing such benefits is still the primary objective of conventional healthcare decision-making.  It is important to recognize, though, that it is difficult to compensate people for being ‘losers’ in healthcare — people would probably find it distasteful to be offered money in return for poorer health, and even if they didn’t, it is impossible to compensate someone who has died as a result of a decision to prioritize others over them.  This makes it difficult to justify an exclusive focus on maximization while disregarding distributional issues when making decisions about how to allocate (inevitably scarce) healthcare resources.

In attempting to incorporate distributional issues (also known as equity or distributive justice) in order to make such decisions ‘fairly’, we must also recognize that the desirability of any particular distribution of resources is a value judgment: what seems fair to one reasonable individual may seem unfair to another.  It is the intrinsic nature of such value judgments that they cannot be resolved by logic alone.  Despite this reality, though, we are still faced with the need to allocate societal resources in the fairest way possible.  I would argue that the most straightforward solution to this problem is to ask people what distribution they would prefer.  This is the basis of the Communitarian approach to resource allocation: resources should be allocated in a way that reflects the preferences of the community.  These preferences determine the objectives of the healthcare system, and the degree to which these preferences are satisfied in turn determines the value the community gets from the healthcare system.

Simple yes/no or rating scale questions are likely to be too simple for eliciting preferences for allocation decisions, most particularly because they do not force respondents to recognize the trade-offs between different allocations.  As discussed in an earlier post, it is crucial to recognize that a decision to give a particular group higher priority necessarily means that another group must receive lower priority.  Such preferences, therefore, are better elicited using discrete choice experiments, or other choice-based stated preference methods such as constant-sum paired comparisons.  Quantitative discrete choice methods were presented by Dean Regier, and if you are interested in more details, his slides can be accessed here.

The Price of Life

Film maker Adam Wishart has made a wonderful documentary about healthcare prioritization that I think illustrates the issues very well.  It focuses on one decision by NICE, the National Institute for Clinical Excellence in the UK, about whether or not the UK healthcare system, the National Health Service (NHS), should fund Revlimid, a very expensive drug used to treat terminally ill patients with myeloma, a cancer of the blood.

It follows a number of patients with terminal myeloma, for whom this decision is literally life or death.  It interviews the developer of the drug, who argues that high drug prices are essential to encourage the innovation that leads to the next generation of lifesaving drugs.  Finally, it shows the dilemma faced by health system administrators who are responsible for allocating a budget between different groups of citizens and patients.  If this expensive drug is funded, it means that they will have to find the money to pay for it elsewhere in their budget.  If you’ve participated in my preferences survey, you’ll recognize their dilemma.

From my point-of-view, a key moment comes at the 48 minute mark of the film, where one of the decision makers expresses the essential truth of healthcare priority setting: if they fund this very expensive drug rather than funding less expensive treatments for patients with different conditions, it implies that an extra year of life lived by myeloma patients is worth more than an extra year of life lived by the other patients. This is the reality of “priority setting”: to give one group higher priority, you have to give another group lower priority.

As mentioned in the film, many people are unhappy with the QALY, or the quality-adjusted life year, as the basis for how these decisions are currently made.  It is assumed that society considers one additional QALY to be equally valuable to everyone, regardless of who gets it and how many they get.  But as the preliminary results of my survey show so far, that’s not how most people feel.  The overwhelming majority of respondents so far have been answering in ways that suggest they do care about who gets that additional QALY.  The objective of my research is to explore how we can make these decisions in a way that we all can agree with. 

I would be very curious to hear your thoughts on this documentary.  If you were the committee chairman at the end, how would you have voted?  Do you accept the need to deny some patients treatment, or is there a different solution where all patients can receive all beneficial treatments?

Public participation and the delusion of objectivity in healthcare decision-making

Much of my research into societal preferences starts with the idea that all members of society should have a say in how healthcare resources are allocated.  However, any discussion around the role of public participation in healthcare decision-making inevitably raises the issue of objectivity — in particular, the idea that ‘decision makers’ have it and the public doesn’t.

The classical definition of objectivity is “having a reality independent of the individual.”  That is, an objective truth should be able to be recognized by everyone without requiring any interpretation or explanation.  For example, the fact that the Empire State building is taller than the tallest NBA basketball player is an objective truth.  Any individual can accept this fact as truth without requiring any explanation or persuasion.  On the other hand, arguing that blue is a better colour than purple is not an objective truth; whether one chooses to accept this as fact depends on some form of persuasion, as well as the particular tastes and perspective of the individual.  The fact that one individual may prefer blue over purple is a subjective truth: it is true from the perspective of that particular individual, but it is not necessarily true for all individuals.

When we as a society need to make an important decision between subjective truths, we conventionally rely on small groups of decision makers who, because of their knowledge, expertise and professionalism, are considered to be uniquely “impersonal, impartial, unbiased and neutral.”  Leaving decisions to impartial, objective decision makers is known as procedural objectivity.  Since the process of making the decision is ‘objective’, it is expected that the result of that decision will also be objective.  That is, the alternative chosen will be objectively better than the alternative not chosen.  ‘Objectively better’ in this context means that a decision can be easily accepted by all individuals, with no further explanation or persuasion.

Like blue versus purple, the ideal allocation of scarce healthcare resources is not an objective truth.  It is not possible to look at two alternative allocations of resources between different groups of patients and say that one is objectively better than the other.  Any individual’s ideal allocation of scarce healthcare resources depends on their preferences: do they prefer to treat younger or older patients; the sickest patients or those most likely to return to full health?  A case can be made for almost any allocation, but as with blue versus purple, the strength of this case ultimately rests upon tastes, perspective and persuasion, not objective truth.  And so healthcare has tended to rely on procedural objectivity and small groups of decision makers to choose the objectively better allocation of resources.

But as Arthur Fine so eloquently puts it, procedural objectivity represents “the view from nowhere, and of no-one in particular.”  By carefully excluding personal perspectives from decisions, we make it impossible to understand the very nature of subjective truths: that truth depends on your perspective!  And to say that such decisions can be objectively better implies that they can be accepted by everyone as truth, with no further explanation or persuasion.   However, it is likely that a cancer patient, told that it will be objectively better for society to no longer fund her drugs, will require a little more explanation and persuasion about this decision.

Fine argues that the fundamental point of procedural objectivity in societal decision-making is not truth, but trust.  People don’t value procedural objectivity because they believe it arrives at a decision that represents an objective truth, they value it because they believe it arrives at a decision they can trust.  In this view of objectivity as trust, objectivity represents anything that improves trust in a decision.  In some cases, trust may be enhanced by the impartiality of professional decision makers.  In other cases, trust may be enhanced by a broader process, with more personal perspectives.  In the case of the cancer patient above, she does not need to believe that such a decision is objectively best, only that she can trust the process by which the decision was made.

It is in enhancing trust that the public has an important role in healthcare decision-making.  Joanna Coast and colleagues asked a group of healthcare decision-makers, including government bureaucrats, physicians, hospital administrators, and members of the general public about the role of citizens in healthcare rationing.  She found that both decision makers and the general public felt that citizens lack sufficient ‘objectivity’ to participate in health care rationing.  Both groups viewed objectivity as the ability to make decisions based solely on facts while setting aside any emotion or empathy.  However, as I suggest above, judgments about the appropriate allocation of healthcare resources rely on subjective values, and cannot be resolved by objective facts alone.

Support for this view of objectivity as trust comes from Mari Broqvist and Peter Garpenby, who asked Swedish citizens about their willingness to accept healthcare rationing.  The feeling of the participants was that citizenship implies a willingness to stand aside for the benefit of others, but also an expectation that others will stand aside when they have greater needs.  However, participants also felt that insufficient knowledge about why some patients were given higher priority would erode their trust in this fragile social contract and make them less willing to stand aside for others.  Broad, public involvement in healthcare decision-making was viewed as a way to enhance understanding and trust.

This suggests to me that concern over the impartiality of citizens in healthcare decision-making is misplaced.  The allocation of healthcare resources is an inherently subjective process and it is not improved by ignoring this truth.  Rather than impartiality, the system depends upon trust, and this trust is best enhanced by the participation of all citizens.  It is not that healthcare decision-making is too important to include citizens; rather, healthcare decision-making is too important not to include citizens.

This post goes a bit beyond my original intent of using this blog simply as a way of communicating survey results to participants.  I found this exercise challenged me to organize my thoughts and present my argument in a clear manner without resorting to (too many) academic references.  It was useful for me, but I am curious to know if you got anything out of it.  Please let me know if I’ve convinced you that citizens have an essential role to play in healthcare decision-making, or if you think there is some glaring hole in my argument.  Is my interpretation of objectivity completely at odds with yours?  Is this the most boring thing you’ve ever read and you want the last 5 minutes of your life back?  I look forward to your comments.

Some early Nova Scotia results

I have been rolling out the survey in stages, and in the first stage most of the invitations to participate went to decision makers and oncologists  here in Nova Scotia.  These invitation were sent out in November and early December, so I expect that most of those that were going to respond have done so by now.   As such, I thought it might by of interest to look at these early results.

Sixty people have responded so far, and as I mentioned, most of these are from Nova Scotia.  It is important to note that these 60 respondents are only a fraction of the total number of respondents I hope to have for the full analysis, but they should be enough to produce a reliable picture of preferences in Nova Scotia.

Respondents were asked to make a choice between two alternative programs, described in terms of different attributes and attribute levels.  The results suggest that patients’ final health state was the single most important factor in people’s choices — the better the final health state, the more likely people were to choose that alternative.  As you can see in the graph below, final health state was almost twice as important as the next most important alternative.

Interestingly, life expectancy before treatment had very little effect on people’s choices.  Although people have a natural urge to help those in need, these results suggest that this urge does not necessarily hold in all circumstances and that it can be moderated by other factors.  This finding, together with the tremendous importance given to final health state, suggests that respondents were less willing to prioritise patients with poor life expectancy who could not be returned to an acceptable quality-of-life.

The number of patients that could be treated was not particularly important to people’s choices.  Respondents were more concerned with the individual characteristics of patients than the aggregate health system benefits.  This may have significant implications for the evaluation of health programs at a population level.  This finding is reinforced by the fact that only 1 out of the 60 respondents chose the alternative that maximised quality-adjusted life years gained in every scenario.

These results are preliminary and it is too soon to say that they present the definitive picture of people’s preferences for how healthcare resources should be allocated in cancer.  It will also be interesting to break these preferences down by the role of the respondent.  For example, are the preferences of decision makers significantly different than those of physicians or the general public?  These answers will come, but will have to wait for a greater number of respondents complete the surveys.

Thank you to all those that have taken the time to complete a survey, and I look forward to updating these results soon.

Economics and the difficult task of prioritization

A friend sent me a link to this engaging talk by Bjorn Lomborg (otherwise known as the Skeptical Environmentalist) on setting global priorities, which I think has relevance to my thesis.  Lomborg is well known, and perhaps infamous, for arguing against addressing Climate Change on the grounds that the economic costs likely outweigh the benefits (see this Wikipedia page for the details).  This conclusion outraged many people who felt that it was inappropriate, and probably impossible, to consider the effects of Climate Change purely in terms of money.  Although he reaches a similar conclusion about the relative priority of addressing Climate Change in the video, I find this version much more compelling because he discusses it in the context of prioritization rather than as an absolute cost-benefit judgment.  Instead of saying we shouldn’t address Climate Change because it’s too expensive, Lomborg argues that addressing Climate Change would gives us less benefit than we could get from spending the same (or even less) money in other areas, such as malaria control and HIV prevention.  Now he’s not talking about sacrificing the environment in order to save some money; he’s saying that in a world of limited resources we should spend our money in ways that give us the most benefit.

This is similar to the problem of prioritizing spending in healthcare.  Most people strongly feel that health is more important than money and object to the idea of “putting a price on someone’s life.”  However, most people also accept that there are competing demands for limited healthcare resources, and because of this there is a need to prioritize some patients over others.  The goal of my research is to discover what characteristics are relevant to people in terms of how patients should be prioritized, and to measure the importance of these characteristics relative to each other.  The ultimate objective is to be able to measure the value we as society get from healthcare — not just as patients, but as citizens.  For example, we may value the fact that our healthcare system will treat the sickest patients first, or those that will receive the greatest benefit from treatment.   From this we can prioritize our limited healthcare resources in ways that society can agree are fair and relevant, and in a way that gives us the greatest overall benefit, without putting a price on anyone’s life.

First results

Before I discuss the results, I want to thank everyone who took the time to complete the surveys and who offered some very helpful comments. These are, indeed, your results!

The main objective of this phase of the research was to explore the characteristics of two different survey designs. Both designs asked respondents to allocate a hypothetical healthcare budget between to alternative programs. Half the respondents were given a discrete choice survey that asked them to allocate the entire budget to their preferred program, while the other half were given a ‘budget pie’ survey that allowed them to allocate budget percentages between the two programs. They could allocate 100% to Program A or Program B, or some combination of the two.

The results…

There is some suggestion in the literature that people may prefer budget pie questions because they do not force an ‘all-or-nothing’ choice as in the discrete choice. However, this may be offset by the more challenging nature of the budget pie tasks, which require respondents to not only identify their preferred alternative, but also judge how strongly they feel about that alternative.  The results are mixed.  People rated the difficulty of both questionnaires to be equal: 12.3% of discrete choice respondents and 12.6% of budget pie respondents said the questionnaires were “somewhat difficult” or “extremely difficult” to understand, and 64.9% of discrete choice respondents and 66.0% of budget pie respondents said the questionnaires were “somewhat difficult” or “extremely difficult” to answer.  Although people were able to understand what was being asked of them, they found it difficult to provide an answer.  This is somewhat reassuring, as it suggests that people were taking the tasks seriously and not just answering at random.  Comments made by respondents also suggested that these tasks helped them appreciate the challenge of making these sorts of decisions in real life.

However, despite the fact that people rated both questionnaires as equally difficult (or equally easy, depending upon how you want to look at it), far more people dropped out of the budget pie questionnaires.  Only 43% of people who started a budget pie questionnaire finished it, compared to 60% of people who started a discrete choice questionnaire.  This suggests that people found the budget pie tasks less acceptable in some regards.  The fact I was only able to collect difficulty ratings from people who completed a questionnaire probably makes the surveys look easier than they were, since the people who found them most difficult would also presumably be the most likely to drop out before rating them.

Interestingly, the two surveys produced different results in terms of which factors were most important to people’s choices.  Here’s the order of importance of the attributes from the discrete choice questionnaires:

  1. The patients’ final health state (better final health state was preferred)
  2. The number of life years gained (more was better)
  3. The patients’ age (younger patients were preferred)
  4. The patient’s initial health state (preferred to treat sicker patients)
  5. The number of patients that would benefit from treatment (more was slightly better)

The patients’ life expectancy without treatment did not at all affect people’s choices in the discrete choice questionnaire; this is referred to as a factor that was ‘not statistically significant’.

And here is the order of attributes from the budget pie questionnaires:

  1. The number of patients that would benefit from treatment (more was much better)
  2. The number of life years gained (more was better)
  3. The patients’ final health state (better final health state was preferred)
  4. The patients’ life expectancy without treatment (preferred patients with greater initial life expectancy)
  5. The patient’s initial health state (preferred to treat sicker patients)
  6. The patients’ age (younger patients were slightly preferred)

All factors were relevant to people’s choices (i.e. were statistically significant).

The different questionnaires clearly focused people’s attention on different aspects of the choice problem.  The budget pie questionnaires, which illustrated the consequences of shifting the budget between programs by increasing or decreasing the number of patients treated and the total life years gained, appeared to focus people’s attention on these two attributes as they were at least twice as important as all the other attributes.   The characteristics of the patients were much less important than the aggregate number of patients treated and life years gained.  On the other hand, respondents to the discrete choice questionnaires, which asked respondents to choose a single preferred program, were much more likely to focus on the characteristics of the patients themselves rather than the overall program.


These results suggest that what factors people say are important in allocating healthcare resources depends upon how we ask them.  Clearly this has important implications for how we can or should go about incorporating societal preferences into healthcare decision-making.  Based on the results presented above, which would you say is more important — the number of patients treated or the patient’s final health state?

These results are so interesting that my committee and I have decided that rather than proceeding with just one design in the larger primary survey, we will proceed with both of them.  This will give us more opportunity to explore why these surveys produce such different results, and which (if either) is a better reflection of people’s true preferences.

Your comments are more than welcome.

Real-life choices in healthcare

This article offers a real-life introduction to some of the issues I’m trying to address in my research. Mostly it highlights the fact that there are no easy answers in how scarce public healthcare resources should be allocated.