Modeling Student Evaluations of Writing and Authors As a Function of Writing Errors

Journal of Language and Education, 6(2), 147-164, 2020. https://doi.org/10.17323/jle.2020.10316

18 Pages Posted: 23 Sep 2020

See all articles by Rod Roscoe

Rod Roscoe

Arizona State University (ASU)

Joshua Wilson

University of Delaware

Melissa Patchan

West Virginia University

Dandan Chen

American Board of Anesthesiology

Adam Johnson

Arizona State University (ASU)

Date Written: June 30, 2020

Abstract

Writers are often judged by their audience, and these evaluations can encompass both the text and the authors. This study built upon prior research on writing evaluation and error perceptions to examine how interconnected or separable are these judgments. Using a within-subjects design, college students evaluated four essays demonstrating no errors, lower-level errors, higher-level errors, or both types. Evaluations included writing quality traits (e.g., conventions, ideas, organization, sentence fluency, and voice) and author characteristics (e.g., creativity, intelligence, generosity, and kindness). Exploratory factor analyses identified latent constructs within these ratings. One construct, Writing Quality and Skill, appeared to combine writing traits and authors’ intellectual ability (e.g., intelligence and knowledgeability). The second construct, Author Personality, seemed to comprise interpersonal author traits (e.g., kindness and loyalty). The two constructs were significantly and positively correlated. These results suggest that students tended to form holistic impressions of writing quality and authors rather than distinct judgments about individual traits. The spillover onto perceptions of authors’ personal characteristics may be representative of latent biases. Student raters were also more sensitive to lower-level errors than higher-level errors. Implications for biases and training related to peer assessment are discussed.

Keywords: college students, writing evaluation, factor analysis, rater bias, writing instruction, peer assessment

Suggested Citation

Roscoe, Rod and Wilson, Joshua and Patchan, Melissa and Chen, Dandan and Johnson, Adam, Modeling Student Evaluations of Writing and Authors As a Function of Writing Errors (June 30, 2020). Journal of Language and Education, 6(2), 147-164, 2020. https://doi.org/10.17323/jle.2020.10316, Available at SSRN: https://ssrn.com/abstract=3669726

Rod Roscoe (Contact Author)

Arizona State University (ASU) ( email )

Farmer Building 440G PO Box 872011
Tempe, AZ 85287
United States

Joshua Wilson

University of Delaware ( email )

Newark, DE 19711
United States

Melissa Patchan

West Virginia University ( email )

PO Box 6025
Morgantown, WV 26506
United States

Dandan Chen

American Board of Anesthesiology ( email )

4208 Six Forks Rd, Suite 1500 Raleigh
NC 27609
United States

Adam Johnson

Arizona State University (ASU) ( email )

Farmer Building 440G PO Box 872011
Tempe, AZ 85287
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
27
Abstract Views
213
PlumX Metrics