International Choice Modelling Conference, International Choice Modelling Conference 2017

Font Size: 
Predicting Behavioral Change using Stated Preferences: Comparison of Preferences Expressed in a Discrete Choice Experiment to Results of a Randomized Controlled Trial on Pedometer-Based Activity Programs
Marcel Bilger, Eric Andrew Finkelstein, Isha Chaudhry

Last modified: 28 March 2017

Abstract


Background

Initially developed in marketing science, Discrete Choice Experiments (DCEs) are being applied in an increasing number of fields, including the environment, transportation, and health. While DCEs can be used to analyze a great variety of issues such as calculating market shares, quantifying the value of non-market goods, and predicting uptake of new services, all applications share a key feature that is intrinsic to DCEs: respondents need to trade between the attributes of the choice tasks they are presented with. The validity of all studies involving DCEs thus critically depends on the validity of the tradeoffs made by the respondents. A well-documented concern is that various hypothetical biases can affect the elicitation of these tradeoffs. The external validity of DCEs is currently under-researched because few settings provide the analyst with data on both revealed and stated preferences simultaneously. In this study, we administrated a DCE to participants of a randomized controlled trial (RCT) to make it possible to compare these two types of data. We focus on one type of hypothetical bias that can adversely affect the validity of DCEs. When an attribute relates to a behavior, respondents might underestimate the effort that is required from them to change their behavior and report unrealistic tradeoffs. While such potential bias can in theory apply to numerous settings, its study is especially relevant to health as respondents might be overly optimistic while stating not only their participation in new public health programs but also the extent of their involvement. Such potential optimism biases warrant to assessment of the external validity of DCEs when they explicitly or implicitly involve behavioral change. In order to conduct such assessments, we administrated a DCE on pedometer-based activity programs at the baseline assessment of a 6-month randomized controlled trial (RCT) comparing four parallel arms: 1) control group, 2) pedometer only, 3) pedometer plus charitable financial incentives, and 4) pedometer plus cash incentives. We hypothesized that DCE respondents will overestimate both their increase in step activity induced by incentives and their participation in the program. The protocol and main results of the RCT are published elsewhere (Finkelstein et al., 2015) and this study focuses on the results from the DCE and comparison with corresponding behaviors observed in the RCT.

Methods

The RCT was conducted in Singapore with a convenience sample of 800 employees from the private sector and government agencies. To be eligible, participants had to be English-speaking, full-time employees,  aged 21 – 65 years, willing to be randomly allocated to one of the four study groups, willing to wear a pedometer for 6 months, able to walk at least 10 stairs continuously, and non-pregnant. All RCT participants were asked to take part in the DCE before randomization into study arms and 755 participants completed the questionnaire.

The DCE mimicked the main features of the activity programs that the participants were about to engage in as part of the RCT. The program attributes were the 1) weekly steps goals to earn the incentive, 2) weekly incentive value, 3) incentive type, 4) value of a one-time enrolment fee. The experimental design was created using Sawtooth (version 8.2.4) using the balanced-overlap option which generates near-efficient designs for measuring main effects while allowing for a reasonable precision for the measurement of interaction terms. The tasks were blocked into 4 versions with 8 tasks per respondent. The survey introduced each program attribute before the DCE tasks. In each DCE task, respondents were asked to first choose between two alternative programs, and then were asked a follow-up question on whether they would enroll in their preferred program if it was offered to them.

All attribute levels were effects-coded and an alternative-specific constant term was included for the opt-out option. The choices made were analyzed using a mixed logit model with normally-distributed parameters to account for preference heterogeneity among respondents. Both the step goals and incentive attributes failed the linearity test and were replaced by quadratic functional forms that were validated by further statistical testing. Trade-offs between step goals and incentive levels were calculated using the estimated parameters of the mixed logit model and these tradeoffs were compared to the increase in step activity observed in the RCT between the cash-incentive and pedometer-only arms.

Main results

In the RCT, we found that, on average, the cash-incentive group walked 16,715 steps more than the pedometer-only group and received an average incentive of $21.7 per week over the 6-month intervention period. In the DCE, we calculated that the willingness-to-accept a weekly cash incentive for walking the same additional 16,715 steps merely amounted to $6.2 (95% CI: 4.0; 8.4). Conversely, the willingness-to-walk that we calculated with the DCE in exchange to the $21.7 weekly cash incentive received in the RCT amounted to a staggering 97,164 (86,730; 107,620) steps per week. In the RCT, the percentage of participants in the cash group that was found to actually use their pedometer was 93% while the uptake we calculated for an identical walking program using the DCE amounted to 97.8% (95.9; 97.7).

Conclusion

We found that when responding to a DCE on pedometer-based activity programs, participants were overestimating their willingness-to-walk in response to incentives by a factor 6 when compared to their actual increase in step activity as measured in a RCT. Participants also slightly overestimated their participation in the program. While other applications might show less discrepancy between stated and observed behaviors, our study casts tremendous doubt on conclusions that are drawn from studies based on stated preferences alone. Studies aiming at predicting behavioral change by means of stated preferences or pursuing other objectives but implicitly involving stated behavioral changes need to simultaneously make a strong case for their external validity in order to produce credible evidence.

References

Finkelstein EA, Sahasranaman A, John G, Haaland BA, Bilger M, Sloan RA, Nang EEK, Evenson KR. Design and baseline characteristics of participants in the TRial of Economic Incentives to Promote Physical Activity (TRIPPA): A randomized controlled trial of a six month pedometer program with financial incentives. Contemporary Clinical Trials 2015;41; 238-247.


Conference registration is required in order to view papers.