Hierarchical Bayes Conjoint Analysis: Recovery of Partworth Heterogeneity from Reduced Experimental Designs

Abstract

The drive to satisfy customers in narrowly defined market segments has led firms to offer wider arrays of products and services. Delivering products and services with the appropriate mix of features for these highly fragmented market segments requires understanding the value that customers place on these features. Conjoint analysis endeavors to unravel the value or partworths, that customers place on the product or service's attributes from experimental subjects' evaluation of profiles based on hypothetical products or services. When the goal is to estimate the heterogeneity in the customers' partworths, traditional estimation methods, such as least squares, require each subject to respond to more profiles than product attributes, resulting in lengthy questionnaires for complex, multiattributed products or services. Long questionnaires pose practical and theoretical problems. Response rates tend to decrease with increasing questionnaire length, and more importantly, academic evidence indicates that long questionnaires may induce response biases.

The problems associated with long questionnaires call for experimental designs and estimation methods that recover the heterogeneity in the partworths with shorter questionnaires. Unlike more popular estimation methods, Hierarchical Bayes (HB) random effects models do not require that individual-level design matrices be of full rank, which leads to the possibility of using fewer profiles per subject than currently used. Can this theoretical possibility be practically implemented?

This paper tests this conjecture with empirical studies and mathematical analysis. The random effects model in the paper describes the heterogeneity in subject-level partworths or regression coefficients with a linear model that can include subject-level covariates. In addition, the error variances are specific to the subjects, thus allowing for the differential use of the measurement scale by different subjects.

In the empirical study, subjects' responses to a full profile design are randomly deleted to test the performance of HB methods with declining sample sizes. These simple experiments indicate that HB methods can recover heterogeneity and estimate individual-level partworths, even when individual-level least squares estimators do not exist due to insufficient degrees of freedom.

Motivated by these empirical studies, the paper analytically investigates the trade-off between the number of profiles per subject and the number of subjects on the statistical accuracy of the estimators that describe the partworth heterogeneity. The paper considers two experimental designs: each subject receives the same set of profiles, and subjects receive different blocks of a fractional factorial design. In the first case, the optimal design, subject to a budget constraint, uses more subjects and fewer profiles per subject when the ratio of unexplained, partworth heterogeneity to unexplained response variance is large. In the second case, one can maintain a given level of estimation accuracy as the number of profiles per subject decreases by increasing the number of subjects assigned to each block.

These results provide marketing researchers the option of using shorter questionnaires for complex products or services. The analysis assumes that response quality is independent of questionnaire length and does not address the impact of design factors on response quality. If response quality and questionnaire length were, in fact, unrelated, then marketing researchers would still find the paper's results useful in improving the efficiency of their conjoint designs. However, if response quality were to decline with questionnaire length, as the preponderance of academic research indicates, then the option to use shorter questionnaires would become even more valuable.

Keywords: consumer preferences, multi-attributed models, consumer research