Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour

Arnulf, J. K., et al. (2014). "Predicting survey responses: how and why semantics shape survey statistics on organizational behaviour." PLoS ONE 9(9): e106361.

13 Pages Posted: 4 Nov 2020

See all articles by Jan Ketil Arnulf

Jan Ketil Arnulf

BI Norwegian Business School

Kai R. Larsen

Leeds School of Business; Information Systems Group; Gallup

Oyvind Martinsen

University of Colorado at Boulder

Chih How Bong

University of Malaysia Sarawak

Date Written: March 26, 2014

Abstract

Some disciplines in the social sciences rely heavily on collecting survey responses to detect empirical relationships among variables. We explored whether these relationships were a priori predictable from the semantic properties of the survey items, using language processing algorithms which are now available as new research methods. Language processing algorithms were used to calculate the semantic similarity among all items in state-of-the-art surveys from Organisational Behavior research. These surveys covered areas such as transformational leadership, work motivation and work outcomes. This information was used to explain and predict the response patterns from real subjects. Semantic algorithms explained 60–86% of the variance in the response patterns and allowed remarkably precise prediction of survey responses from humans, except in a personality test. Even the relationships between independent and their purported dependent variables were accurately predicted. This raises concern about the empirical nature of data collected through some surveys if results are already given a priori through the way subjects are being asked. Survey response patterns seem heavily determined by semantics. Language algorithms may suggest these prior to administering a survey. This study suggests that semantic algorithms are becoming new tools for the social sciences, opening perspectives on survey responses that prevalent psychometric theory cannot explain.

Keywords: Leadership, Latent Semantic Analysis, Survey Research, Semantic Theory of Survey Response, Five-Factor Personality, Transformational Leadership

Suggested Citation

Arnulf, Jan Ketil and Larsen, Kai R. and Martinsen, Oyvind and Bong, Chih How, Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour (March 26, 2014). Arnulf, J. K., et al. (2014). "Predicting survey responses: how and why semantics shape survey statistics on organizational behaviour." PLoS ONE 9(9): e106361., Available at SSRN: https://ssrn.com/abstract=3716118

Jan Ketil Arnulf (Contact Author)

BI Norwegian Business School ( email )

Nydalsveien 37
Oslo, 0442
Norway

Kai R. Larsen

Leeds School of Business; Information Systems Group ( email )

995 Regent Dr.
Boulder, CO 80309-0419
United States

Gallup ( email )

901 F St NW
Washington, DC 20004
United States

Oyvind Martinsen

University of Colorado at Boulder ( email )

1070 Edinboro Drive
Boulder, CO 80309
United States

Chih How Bong

University of Malaysia Sarawak ( email )

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
11
Abstract Views
155
PlumX Metrics