International Choice Modelling Conference, International Choice Modelling Conference 2017

Font Size: 
IMPACT AND DETECTION OF STRAIGHTLINING RESPONSE IN ONLINE SURVEYS
Peter Grant Moffatt

Last modified: 28 March 2017

Abstract


In the context of a stated-binary-choice experiment, straightliners (Maronick, 2009) or satisficers (Krosnick, 1991), are survey participants whose responses are not related to the attributes of the two alternatives, and can therefore be perceived as choosing at random in every choice task that they face.  It is useful to be able to identify the presence of straightliners in a sample, for two reasons.  Firstly, their responses can be removed from the sample before the main data analysis commences, avoiding the often severe bias in estimation arising when they are included.  Secondly, identified  straightliners can (if this is feasible) be excluded from future surveys, hence sparing wasteful use of survey resources.

This paper first illustrates the impact of straightlining response my means of a monte-carlo study of a straightforward stated choice model with two attributes (travel cost and travel time).  The data generation process is based on the following utility function for travel mode s:

U_s = -2 time_s – cost_s + e_s         (1)

e_s ~ N(0.0.5)

Equation (1) implies that the value of time is exactly 2.0 for all individuals.  There are n=200 simulated respondents, each facing T=10 binary choice problems containing two alternatives with different time-cost combinations.  A proportion of straightliners, p, is included in the sample.  At each replication, the marginal disutilities of these attributes are estimated using probit ML, and an estimate of the value-of-time deduced.  As expected, both of the marginal disutilities are biased towards zero in the presence of straightliners.  It is also shown that the estimate of value-of-time is biased upwards, and much less precise, in the presence of straightliners (see table below).  For example, if straightliners make up 20% of the sample, the estimate of value of time will be biased upwards by more than 50%, and its standard deviation is increased by a factor of 4.

p:                         0.00  0.05  0.10  0.15  0.20

mean value of time: 2.02  2.31  2.56  2.80  3.04

s.d. value of time:   0.21  0.32  0.44  0.57  0.80

In the second part of the paper, we develop a finite mixture model in which the proportion of straightliners in the population can be estimated, and hence their presence in the sample can be tested for and adjusted for.  Thirdly, we consider ways of allowing the straightlining propensity to depend on individual characteristics.  Fourthly, we construct formulae for the posterior probability of each individual sample member being a straightliner.  This is the formula that is used to identify straightliners, with a view to excluding their responses from estimation, and (if possible) debarring them from participation in future surveys.

We then extend the model to allow certain respondents make considered choices at the start of the sequence, but at a certain point to switch over to being a straightliner.  Finally, we consider ways of assessing the impact of the participation fee.  The obvious issue to be addressed here is the extent to which an increase in the participation fee can bring about an increase in the number of tasks for which considered choices are made.

 


Conference registration is required in order to view papers.