Identifying Good Ideas: Constructs and Scales for Idea Evaluation

Journal of Association for Information Systems, Vol. 7, No. 10, pp. 646-699, 2006

55 Pages Posted: 5 Jun 2009 Last revised: 5 Nov 2014

See all articles by Douglas L. Dean

Douglas L. Dean

Brigham Young University - Information Systems Department

Jill Hender

University of Reading - Department of Management

Tom Rodgers

Texas A&M University

Eric Santanen

Bucknell University

Date Written: 2006

Abstract

Researchers and practitioners have an abiding interest in improving tools and methods to support idea generation. In studies that go beyond merely enumerating ideas, researchers typically select one or more of the following three constructs, which are often operationalized as the dependent variable(s): 1) idea quality, 2) idea novelty, which is sometimes referred to as rarity or unusualness, and 3) idea creativity. It has been chronically problematic to compare findings across studies because these evaluation constructs have been variously defined and the constructs have been sampled in different ways. For example, some researchers term an idea 'creative'' if it is novel, while others consider an idea to be creative only if it is also applicable, effective, and implementable. This paper examines 90 studies on creativity and idea generation. Within the creativity studies considered here, the novelty of ideas was always measured, but in some cases the ideas had to also meet additional requirements to be considered creative. Some studies that examined idea quality also assessed novelty, while others measured different quality attributes, such as effectiveness and implementability, instead. This paper describes a method for evaluating ideas with regard to four dimensions - novelty, workability, relevance, and specificity - and has identified two measurable sub-dimensions for each of the four main dimensions. An action-research approach was used to develop ordinal scales anchored by clearly differentiable descriptions for each sub-dimension. Confirmatory factor analysis revealed high loadings among the sub-dimensions that comprise each dimension as well as high discriminant validity between dimensions. Application of this method resulted in high inter-rater reliability even when the method was applied by different raters to different problems and to ideas produced by both manual methods and group support systems (GSS).

Keywords: idea evaluation, brainstorming, creativity, idea generation, group support systems, measurement

Suggested Citation

Dean, Douglas L. and Hender, Jill and Rodgers, Tom and Santanen, Eric, Identifying Good Ideas: Constructs and Scales for Idea Evaluation (2006). Journal of Association for Information Systems, Vol. 7, No. 10, pp. 646-699, 2006, Available at SSRN: https://ssrn.com/abstract=1413393

Douglas L. Dean (Contact Author)

Brigham Young University - Information Systems Department ( email )

786 Tanner Building
Marriott School
Provo, UT 84602
United States
801-422-3247 (Phone)
801-422-0573 (Fax)

HOME PAGE: http://isys.byu.edu/

Jill Hender

University of Reading - Department of Management ( email )

United Kingdom

Tom Rodgers

Texas A&M University ( email )

Langford Building A
798 Ross St.
College Station, TX 77843-3137
United States

Eric Santanen

Bucknell University ( email )

701 Moore Ave.
Lewisburg, PA 17837
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
1,889
Abstract Views
5,261
Rank
16,294
PlumX Metrics