International Choice Modelling Conference, International Choice Modelling Conference 2017

Font Size: 
Measuring treatment preferences of people with type 2 diabetes with a discrete choice experiment and best-worst scaling: a randomized experiment
John FP Bridges, Ellen Margreet Janssen

Last modified: 28 March 2017


Objectives: An increasing number of researchers, policy makers, and clinicians are interested in measuring patient preferences for use in patient centered research and medical decision making. Discrete choice experiments (DCEs) are used most often to evaluate preferences in health and clear guidelines have emerged for applying them in a healthcare setting. DCEs can be difficult to implement due to complex experimental designs and analytical approaches. Best-worst scaling (BWS) case 2 (i.e. the profile case) offers an alternative approach that allows for the use of simple experimental designs and analytical approaches. This said, BWS might suffer from floor and ceiling effects and the exploration of tradeoffs across attributes (although empirically possible) is less intuitive and not necessarily grounded in theory. Few studies have compared DCE and BWS approaches. We sought to compare the concordance of preference estimates, subjective views of the choice tasks, respondent burden, and underlying preference heterogeneity between DCE and BWS targeted at measuring patient preferences for type 2 diabetes treatment.

Methods: A randomized control trial ( Identifier: NCT02637622) was conducted among a nationally representative, and racially/ethnically diverse, sample of patients with type 2 diabetes in the United States. Participants made choices surrounding 6 treatment attributes that were chosen and developed into choice tasks using an iterative process of evidence synthesis, expert consultations, stakeholder engagement, pretest interviews, and pilot testing. Participants were randomly assigned to complete one of three blocks of 16 paired-comparison DCE choice tasks (identified using a Bayesian D-efficient design) or 18 BWS choice tasks (identified using a main effect orthogonal design). Preference results were analyzed using conditional logit (CL) and mixed logit (MXL) models. Estimates of the DCE were rescaled to account for the difference in the scale factor between the DCE and BWS. We also compared objective and subjective measures of respondents’ understanding of the tasks and associated response burdens using self-reported evaluation questions and response time.

Results: 552 respondents were randomly assigned to the DCE and 551 to the BWS case 2 (response rate 66%). Participants that completed the DCE and BWS did not differ on demographic and socioeconomic characteristics, health status, and disease history characteristics (p=>0.05). Overall, the DCE took less time to complete than the BWS, with a mean time to complete all tasks of 13.1 and 14.6 minutes respectively (p = 0.04). Mean time to complete an individual choice task did not differ between the DCE and BWS at 48.7 and 49.1 second respectively (p = 0.9). DCE participants were more likely to (strongly) agree that the choice tasks were easy to understand (p=0.028). There were no differences in how the DCE and BWS were evaluated on ease of answering or ability to consistently elicit preferences (p>0.05). Preference estimates from the DCE and BWS were highly correlated both for the CL and the MXL models (pearson’s rho > 0.88). The relative variance scale factor was more than two times larger (indicating a smaller error variance) for the BWS than for the DCE. Estimated preference weights for the DCE and BWS were statistically different for most attribute levels (p<0.001 for most levels) for both the CL and the MXL models. In the MXL, standard deviations associated with attribute levels were the same for the DCE and BWS (p>0.05) for all but two of the attribute levels. Standardized relative attribute importance scores were different for each attribute (p<0.01) except for nausea (p=0.08). Overall, participants assigned more value to treatment benefit (39%) in the BWS than in the DCE (28%).

Conclusions: Results from the BWS and DCE were highly correlated but BWS resulted in preference estimates that were more consistent than those from the DCE. The slightly better understanding of the DCE choice tasks that was observed might relate to people’s familiarity with choosing between two goods or services in their daily life, while choosing the best and the worst aspect of a good or service might be less intuitive. Exploration of lexicographic preferences in the DCE indicated that 13.2% of participants showed attribute dominance for treatment harm attributes and only 1.8% showed attribute dominance for treatment benefit. Simplifying heuristics might therefore have contributed to the higher relative importance assigned to treatment harm in the DCE. In addition, correlations between treatment attributes in the DCE and BWS experimental designs need to be further examined to understand whether observed preference differences might be due to underlying mulitcolinearity. Given that BWS surveys can be easier to implement in terms of experimental design and data analysis, might suffer less from lexicographic preferences, and might result in more pronounced preferences, researchers should consider this stated-preference method when eliciting patient preference information.

Conference registration is required in order to view papers.