International Choice Modelling Conference, International Choice Modelling Conference 2017

Font Size: 
Aggregation Biases in Discrete Choice Models
Timothy Wong, DAVID BROWNSTONE, David Bunch

Last modified: 28 March 2017

Abstract


This paper studies the practice of aggregating choices within discrete choice models. Researchers often do not observe choices at the exact level they are made, and hence aggregate choices to the level that is observed. Modeling choices at a fine level of detail can also lead to large choice sets that exceed the practical capabilities for model estimation. However, the practice of aggregation miss-specifies the true choice set of interest. Previous work (Brownstone and Li, 2014 and Wong, 2015) investigated this concern within the context of the Berry, Levinsohn, and Pakes (BLP) choice model for micro- and macro-level data. This paper compares all of the commonly used methods for estimating choice models where choices need to be aggregated. In addition to the aggregation methods given by McFadden (1978) and Brownstone and Li (2014), we also investigate the common practices of averaging attributes across the aggregated alternatives or choosing one alternative to represent the alternatives being aggregated.

There are many applied choice applications where aggregating choices is a common issue.  Automobile type choice models typically model choice at the vehicle type level (e.g. small car, large car, SUV, truck, etc.), but in the U.S. there are approximately 1300 different vehicle configurations available in the new vehicle market.  However all of the major surveys used for vehicle choice modeling in the U.S. (the National Household Travel Survey (NHTS) or the Consumer Expenditure Survey) only collect information at the vehicle make/model level (e.g. Honda Civic).  There are about 230 distinct make/models in the current U.S. new vehicle market, so using these data sources implies considerable aggregation of choices.  For example, there are 7 different trim lines under the Honda Civic label, and over 100 trim lines under the Ford F-150 make/model label.

We begin by considering the simplest possible Monte Carlo setting to isolate the impact of aggregating alternatives from possible model misspecification.  The data generation process is specified as conditional logit with an outside good, 3 make/models grouped into “cars” and 3 make/models grouped into “trucks.” There are 98 distinct alternative vehicles that are grouped into make/models.  In one case there was only one vehicle assigned to a particular make/model which corresponds to some vehicles in the real marketplace (e.g. Toyota Prius in the early 2000s), and the remaining make/models correspond to between 2 and 55 vehicles. We consider all of the methods in the literature for estimating the 7-alternative choice model corresponding to only observing the make/model.  As is true in the real U.S. vehicle market we assume that there are data on the attributes of each vehicle, but unlike BLP we do not assume that we have any market share information for the vehicles. McFadden’s (1978) aggregation procedure only requires information on the mean and covariances of the attributes being aggregated, but it is only valid for aggregating alternatives at the lowest level of a Nested Logit choice model. We examine the performance of the estimators with sample sizes ranging from 500 to 10,000.  This range encompasses most of the applied literature.

The Monte Carlo results show that the only method that performs well for all sample sizes is the “broad choice” estimator described in Brownstone and Li (2014).  This is not surprising since this estimator is the maximum likelihood estimator for this problem.  Both the coefficient estimators and their covariance estimators are biased for the other methods. Simply averaging attributes clearly leads to measurement error, and this is not helped by including the logarithm of the number of vehicles being aggregated as is done in some studies.  McFadden’s (1978) method would be consistent if the joint distribution of the attributes that are aggregated are multivariate normal, but this approximation is not satisfied in our Monte Carlo design even if we relax the implied constraints on the parameters.

Finally we apply the estimation procedures used in the Monte Carlo design to real vehicle choice data.  We use data from the 2009 NHTS survey supplemented by detailed data on attributes for each of the 1120 2009 model year vehicles.  We do not include an outside good since it would include both purchasing used vehicles and not buying any vehicles.  Since the NHTS only collects data on make, model, and year for each vehicle, these 1120 vehicles need to be aggregated into 235 make/model classes.  As expected from the Monte Carlo results, we find large differences between the estimates produced from the various methods.  The confidence bands for willingness to pay estimates do not overlap, and the Broad Choice and McFadden’s method yield larger willingness to pay estimates than averaging attributes or choosing representative vehicles.

This paper shows that it is critical to properly account for the biases introduced when aggregating alternatives.  We have demonstrated the importance of these biases in both a Monte Carlo study and an empirical example using the conditional logit model in the simplest case where there is no external market share data available. Our earlier work shows that incorporating external market share data does improve the quality of the estimates, but does not alleviate the problems caused by aggregating alternatives. The Broad Choice maximum likelihood method is the only one that performs well in our Monte Carlo study, and we expect that it will continue to perform well in other applications with more flexible discrete choice models.  The simple expedients of averaging attributes or picking a “representative” alternative perform very poorly in our studies, and we expect them to perform at least as poorly in other situations.

 

References:

Brownstone, D. and P. Li. A Model for Broad Choice Data, Working Paper, Department of Economics, University of California, Irvine, February, 2014.

McFadden, D., 1978. Modeling the choice of residential location. In A. Karlqvist, L. Lundqvist, F. Snickars, and J.Weibull (Eds.), Spatial Interaction Theory and Planning Models. North-Holland, Amsterdam: 75-96.

Wong, T. J. Econometric Models in Transportation, Ph.D. thesis, Department of Economics, University of California, Irvine, June, 2015.

 


Conference registration is required in order to view papers.