Making the Most Out of Your Factorial Survey. Alternative Methods for the Presenting and Analyzing Data Gathered in Within- and Between-Subjects Design Factorial Surveys

Abstract

Factorial surveys became an increasingly popular method in various disciplines of social research, ranging from psychology to sociology to behavioral economics. By combining elements of experiments and surveys, the method allows to study more complex relationships, combining experimental stimuli, personal features, and situational elements. There is a range of established statistical methods to analyze the data obtained from such studies, which focus on drivers of individual reactions to the vignette and its features. We offer and discuss alternative approaches, which allow for a different view on the information obtained from factorial surveys by focusing on the level of the vignette. These approaches allow to identify the latent dimensions underlying the multitude of situational facets chosen by the researcher when designing the vignette universe, to see which vignettes are actually similar to each other, regardless of their content, and to identify factors which structure the underlying space of the vignette universe, which may or may not correspond to the intentions underlying the construction of the vignette universe. Practical cases which construct an ethical space underlying a factorial survey from compliance and ethics research illustrate the alternative methods and their results.

Keywords: Factorial Surveys, Experimental Research Designs, Statistical Analysis, Data Presentation