International Choice Modelling Conference, International Choice Modelling Conference 2017

Font Size: 
Selecting a study sample for discrete choice experiments: the effect of psychological distance
Jorien Veldwijk, Karin Groothuis-Oudshoorn, Ulrik Kihlbom, Sophie Langenskiold, Evelien Dekker, Frank Kallenberg, Ardine de Wit, Mattijs Lambooij

Last modified: 28 March 2017

Abstract


Background: Discrete Choice Experiments (DCE) are increasingly being used in health care and public health to inform policy makers by calculating welfare measures such as willingness to accept risks or willingness to pay. Since DCEs are used as input for policymaking, the accuracy and validity of the measured (i.e., stated) preferences is essential; measured preferences should accurately reflect the true preferences of the actual target population of the intervention at hand. In some health care related situation, but particularly in the public health setting, defining the target population and recruiting a study sample for DCEs is sometimes not that clear-cut. For some newly designed treatments or preventive initiatives, the exact target population might not exist (yet). Several DCEs on public health initiatives are therefore conducted among a sample of the general population. In contrast to patients, the general population might be less experienced in making health-related decisions. Moreover, they might experience problems imagining to take decisions about situations they are unfamiliar with and which do not apply to them at this point in their life. Eliciting preferences from the relatively inexperienced general population for a specific public health initiative that might become available in the future, potentially leads to biased estimates. Several theories suggest that awareness, risk perception, experience and time to event influence individuals’ attitudes, preferences and intention. In particular the construal level theory describes that individual’s preferences depend on their perceived psychological distance regarding the decision they have to make. Psychological distance is defined on several dimensions, among which temporal (time to the event) and hypothetical (experience with the event). Based on the distance people perceive toward the decision they have to take, their preferences are likely to differ. Purpose: To investigate to what extent the outcomes of a DCE differ due to the psychological distance of respondents towards the decision at hand by selecting different study samples. Methods: A D-efficient DCE questionnaire was designed based on a panel mixed-multinomial-logit model, including beta priors from a pilot study and assuming all attributes would be effects coded. The DCE consisted of nine unique choice tasks each containing two alternatives and was distributed among two populations: (1) a representative sample of the Dutch general population aged 55-65 years, whom had not yet participated in the National colorectal cancer (CRC) screening program and (2) a sample of Dutch individuals who attended an information meeting about colonoscopies since blood was detected in their stool during the CRC screening program. The DCE consisted of four attributes related to the decision whether to participate in genetic screening for CRC: risk of being genetically predisposed, risk of developing CRC, frequency of follow-up colonoscopies and survival. Direct attribute ranking, dominant decision-making behavior, preference heterogeneity and relative importance weights (based on panel-MIXL) were compared between the two populations. Differences in attribute level estimates and the role of the scale parameter were determined with the Swait and Louviere test. Results: The proportion of respondents who ranked survival to be the most important attribute (63% versus 56%) and who showed dominant decision-making behavior for this attribute (13.8% versus 9.2%) was significantly higher in the screened population versus the general population. In both populations significant preference heterogeneity was shown for three out of the four attributes (risk of developing CRC, frequency of follow-up colonoscopies and survival). Although survival was the most important attribute for decision-making in both populations, the relative distance between the importance weights of the attributes differed between the populations. Finally, the log likelihoods of the MNL model fitted in within both datasets separately (general population: -2897.5, screened population: -1041.4) were tested against the log likelihood of the MNL model for the pooled dataset (-4310.5). The MNL model of the pooled dataset accounted for the scale parameter, which was estimated at 1.22. By means of a Chi-square test the hypothesis of equal attribute level estimates between both datasets was rejected. The differences in attribute level estimates between the general and the screened population are statistically significant. Conclusion: This study shows that selecting different target populations for the same DCE yields different study outcomes. Based on the psychological distances towards the decision at hand respondents reported differences in direct attribute ranking, dominant decision-making behavior, relative importance scores and attribute level estimates. People who are at higher risk of being diagnosed with CRC (screened population) and who thereby are more close to the actual decision making phase, have different preferences compared to members of the general population. Results of this study are in line with the assumptions of the construal level theory. Decisions that are more distant, are construed on a higher level and abstract notions of the event, while decisions that are more closely situated, are construed on more concrete features of the decisions. Specifically, methods need to be explored to reduce the effect of psychological distance of preferences when selecting study samples for a DCE. Of course selecting the adequate population is vital, but if the target population does not exist yet, other measures should be developed to reduce the psychological distance the alternative population perceives towards the decision. For instance using refined educational tools (such as serious games (scenario-based educational tools)) might help to improve respondents’ ability to imagine themselves to be in a particular situation. Additionally, future research should be conducted to determine whether the effects of psychological distance on preferences is constant over different decision situations, disease areas and respondent populations. If that is the case, researchers who are conducting a DCE regarding a new treatment or preventive initiative for which the actual target population is not fully defined (yet), can at least report something about the potential bias of their results due to the increased psychological distance of their study sample. In conclusion, this study shows the importance of adequate sample selection. It is therefore advocated to increase attention to target population selection and reporting in future DCE studies.

Conference registration is required in order to view papers.