Observed Efficiency of a D-Optimal Design in an Interactive Agency Choice Experiment
Last modified: 15 March 2009
Abstract
There have been a number of recent calls within the choice literature to examine the role of social interactions upon preference formation. McFadden (2001a,b) recently stated that this area should be a high priority research agenda for choice modelers. Manski (2000) has also came to a similar conclusion and offered a plea for better data to assist in understanding the role of interactions between social agents. The interactive agency choice experiment (IACE) methodology represents a recent development in the area of discrete choice directed towards these pleas (see e.g., Brewer and Hensher 2000). The study of the influences that group interactions have upon choice bring with them not only issues that need to be overcome in terms of modeling, but also in terms of setting up the stated choice experiment itself.
Currently, the state of practice in experimental design centres on orthogonal designs (Alpizar et al., 2003), which are suitable when applied to surveys with a large sample size. In a stated choice experiment involving interdependent freight stakeholders in Sydney (see Hensher and Puckett 2007, Puckett et al. 2007, Puckett and Hensher 2008), one significant empirical constraint was difficulty in recruiting unique decision-making groups to participate. The expected relatively small sample size led us to seek an alternative experimental design. That is, we decided to construct an optimal design that utilised extant information regarding the preferences and experiences of respondents, to achieve statistically significant parameter estimates under a relatively low sample size (see Rose and Bliemer, 2006).
We chose to optimise with respect to the criterion of D- efficiency when generating the experimental design, given that it is less computationally burdensome to generate a D-efficient design than either an A-efficient and G-efficient design (Kuhfeld et al., 1994) and that A- and G-efficiency are highly correlated with D-efficiency (Carlsson and Martinsson, 2003). A D-efficient design is one of the myriad candidate D-optimal profiles that satisfies a desired level of statistical efficiency by minimising the expected standard errors of resulting marginal utility parameters. The D-efficient design utilised in the experiment was derived by utilising prior information gathered through a literature review, previous studies and a pilot study within an iterative optimisation technique.
The experimental design developed for the study is unique, in that it centred on the choices of interdependent respondents. Hence, the generation of the design had to account for the preferences of two distinct classes of decision makers: buyers and sellers of road freight transport. This paper discusses the process by which these (non-coincident) preferences were used to seed the generation of the experimental design, and then examines the relative success of the design. Moving beyond the functional result (i.e., the design did indeed achieve significant parameter estimates under a small sample size), this paper demonstrates the efficiency of the observed choice data relative to the design. We then highlight lessons learned from the study with respect to optimal design, with a focus on strategies to maximise the efficiency of experimental designs for interactive choice settings.
References
Alpizar, F., Carlsson, F., and Martinsson, P. 2003, “Using choice experiments for non-market valuation”, Economic Issues, vol. 8, 83-110.
Bliemer, M.C. and Rose, J.M. (2006) Designing Stated Choice Experiments: The state of the Art, accepted for presentation at 11th International Conference on Travel Behaviour Research - Kyoto, August 16-20, Japan.
Brewer, A. and Hensher, D. A. (2000) Distributed work and travel behaviour: the dynamics of interactive agency choices between employers and employees, Transportation, vol. 27, no. 1, 117-148.
Carlsson, F. and Martinsson, P. 2003, “Design techniques for stated preference methods in health economics”, Health Economics, vol. 12, no. 4, 281-294.
Hensher, D.A. and Puckett, S.M. (2007) Power, concession and agreement in freight distribution chains subject to distance-based user charges, International Journal of Logistics: Research and Applications vol. 11, no. 2, 81-100.
Kuhfeld, W., Tobias, R. and Garrat, M. 1994, “Efficient experimental design with marketing research applications”, Journal of Marketing Research, vol. 31, 545-557.
Manski, C. F. (2000) Economic analysis of social interactions, Journal of Economic Perspectives, vol. 14, no. 3, 115-136.
McFadden, D. (2001a), Economic choices, American Economic Review, vol. 91, no. 3, 351-378.
McFadden, D. (2001b) Overview of the Invitational Choice Symposium, Asilomar Conference Center, California, June.
Puckett, S.M. and Hensher, D.A. (2008) The role of attribute processing strategies in estimating the preferences of road freight stakeholders, Transportation Research Part E, vol. 44, 379-395.
Puckett, S.M., Hensher, D.A., Collins A. and Rose, J. (2007) Design and development of a stated choice experiment in a two-agent setting: interactions between buyers and sellers of urban freight distribution services, Transportation, vol. 34, no. 4, 429-451.
Full Text: DOC