Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey

Abstract

This paper investigates how expected and actual questionnaire length affects cooperation rates and a variety of indicators of data quality in web surveys. We hypothesized that the expected length of a web-based questionnaire is negatively related to the initial willingness to participate. Moreover, the serial position of questions was predicted to influence four indicators of data quality. We hypothesized that questions asked later in a web-based questionnaire will, compared to those asked earlier, be associated with (a) shorter response times, (b) higher item-nonresponse rates, (c) shorter answers to open-ended questions, and (d) less variability to items arranged in grids. To test these assumptions, we manipulated the stated length (10, 20, and 30 minutes) and the position of questions in an online questionnaire consisting of randomly ordered blocks of thematically related questions. As expected, the longer the stated length, the fewer respondents started and completed the questionnaire. In addition, answers to questions positioned later in the questionnaire were faster, shorter, and more uniform than answers to questions positioned near the beginning.