International Choice Modelling Conference, International Choice Modelling Conference 2015

Font Size: 
Uncertainty in Choice Experiments: Will Eye-Tracking provide useful measures?
Kennet Uggeldahl, Catrine Jacobsen, Thomas Lundhede, Søren Bøye Olsen

Last modified: 11 May 2015


In stated preference studies, respondents are assumed to answer valuation questions in a way that precisely reflects their true preferences. Several studies have, however, found that respondents in stated preference studies indicate that they are uncertain about their preferences for the good, and that ignoring this uncertainty might bias the value estimates or at least the variance estimates and hence the inference and conclusions obtained from valuation studies (Li and Mattson, 1995). Previous attempts at accounting for uncertainty have predominantly relied on either respondent statements of uncertainty in choice or model-based inferred uncertainty. However, these approaches are sensitive to modelling assumptions as well as differences in individual respondents' use of certainty scales, and endogeneity issues are often looming. In this paper, we test whether eye-tracking may provide a better measure of preference uncertainty in stated choice experiments– one that is objective and exogenous.

A total of 193 respondents answered a food choice experiment about minced meat packages varying in environmental and ethical labelling. Respondents were placed in front of a screen with an eye-tracking device which recorded their eye movements during the process. The eye-tracker sampled the respondent’s eyes 60 times a second, measuring e.g. gaze location and pupil size. The data was used to construct a variable accounting for the degree of switching between alternatives, which was used as an exogenous measure of observed uncertainty. A switch between alternatives was defined as the respondent fixating, i.e. looking at one alternative for more than 100 milliseconds, followed by a fixation on another alternative. This was based on findings from cognitive science indicating that the degree of switching between alternatives is correlated with stated certainty (Jacobsen, 2014). To verify this finding, respondents were also asked to state their self-perceived subjective certainty in choice after each valuation question.

Using respondents’ stated certainty in choice as the dependent variable in a regression model we find that increased switching between alternatives in a choice set indicates a lower stated certainty in choice. Also longer response times, a bigger absolute difference in gaze time between alternatives and the price of the chosen alternative are related to lower stated certainty in choice. As expected, larger absolute utility difference between alternatives, based on a random parameters logit model, indicates higher stated certainty in choice.

Following Lundhede et al. (2009), we account for uncertainty in the choice model by parameterisation of the scale function. Parameters for the function were defined based on the variables found to explain stated certainty in choice. We find that there is a significant difference in the scale between segments defined based on either the degree of switching, response time or stated certainty. Choice observations where switching is higher than the base level, have a lower scale which means that the variance in the error term is bigger for this group. As a higher degree of switching was found to indicate more uncertain answers, the larger variance in the error term is likely an expression of uncertainty. A longer response time was found to have a smaller scale compared to the base level, which is consistent with the findings that longer response times indicate more uncertain answers.

Using respondents’ statements of certainty in choice as a variable explaining the variance in the error term indicated that choice observations with lower stated certainty than the base level had a lower scale and thus higher variance.

The best performing model used respondents’ stated certainty directly to account for scale heterogeneity, followed by the models using response time and the degree of switching between alternatives. The differences between the models were, however, quite small, with adjusted pseudo-R2 values ranging between 0.41 and 0.42.There were no significant differences between the mean WTP estimates across the models, but small differences in the variance of the WTP were found.

These findings seem to indicate that the degree of switching between alternatives, obtained with eye-tracking, can be used as an exogenous measure of observed uncertainty in modeling preference uncertainty in the choice model. However, as response time, which could also be considered an exogenous and objective measure of choice certainty, yields similar results, the effort associated with setting up an eye-tracking experiment in order to obtain the switching measure might not be justified on this basis alone.




Jacobsen, C. (2014): It Is All In Your Eyes: An Eye-Tracking Study of Confidence in Value-Based Choice. Working Paper.

Li, C. & L. Mattsson (1995): Discrete Choice under Preference Uncertainty: An Improved Structural Model for Contingent Valuation, Journal of Environmental Economics and Management, vol. 28, no. 2, pp. 256-269.

Lundhede, T. H., Olsen, S. B., Jacobsen, J. B., & Thorsen, B. J. (2009). Handling respondent uncertainty in choice experiments: evaluating recoding approaches against explicit modelling of uncertainty. Journal of Choice Modelling, 2(2), 118-147.

Conference registration is required in order to view papers.