Behizadeh, Nadia & George Engelhard Jr. “Historical View of the Influence of measurement and writing theories…” Assessing Writing 16 (2011) (22 pages)
Behizadeh and Engelhard offer a historical perspective that specifically looks at the constituent fields of study that influence the developments of writing assessments. As they write, writing assessment can be viewed as “products of interactions over time between writing theories, measurement theorists, test makers, teachers, and administrators” (190). Since the measurement of writing ability and instruction necessarily influences the quality and content of writing instruction and ability (impact & washback), we need to develop theoretically sound assessments; however, the question of which theory is sound is often contested. The authors look, then, historically at the ways that measurement theory and writing theory have (or have not) influenced writing assessment. Writing assessment, then, emerges through and is influenced by the interaction between both measurement and writing theory.
In measurement theory, the authors point to the pendulum of the test score research tradition (marked by a concern with measurement error) and scaling theory research tradition (marked by observing individual responses as a way to make wider, group decisions). In writing theory, the authors observe three research traditions beginning with (1) writing as form—including mechanics, grammar, and isolated skills; (2) writing as ideas and content—including creative solutions, applied skills to authentic situations, and poetic, new or thought-provoking content; and (3) writing as a socially and culturally contextualized process.
1900-1920s: “writing assessments in the U.S. appears to have been primarily nested within classrooms, and there was a push for objective classroom assessments of writing ability” including the implementation of Hillegas’ scale in classroom settings (195). There is no evidence of writing theories affecting writing assessment or measurement theories during this time—likewise, no influence of measurement theories influencing writing theories. However, Thorndike was developing scales for assessing writing at this time; thus, measurement theory was only emergent in writing assessments.
1930-1940s: “writing assessments were strongly influenced by measurement theory as demonstrated by the dev’t of multiple-choice tests aligned with test-score theory and continuing focu son estimating reliability coefficients and error variances” (198).
1950-1960s: “measurement theory had the greatest influence on writing assessment practices….In addition, although there is not clear evidence, measurement theory possibly influenced writing theory, which would partially explain the triumph of form. With the test-score tradition dominant, writing theories may have been pressured to conduct research with easily quantifiable outcomes…writing assessments themselves were influencing writing theories” (200).
1970-1980s: “assessment practices still were dominated by the use of tests composed of multiple choice tests. Writing theory changed dramatically during this period, while measurement and writing assessments changed very little, highlighting the weak influence writing theory had on writing assessments in the U.S.” (202).
1990-present: During this period, there was a need to resolve the tension between the psychometric community and writing community. The emergence of portfolio as a means of assessment was due to the involvement of teachers of writing rather than solely assessment specialists. “Writing theory and writing assessment theory influencing writing assessment was a clear reversal of the dominant trends” of measurement theory “until this time” (205).
As the authors conclude, there was no “significant evidence that writing theory influenced writing assessment…by far the strongest influence throughout the past century has been measurement theory on writing assessment practices“ (205). The gap between writing theories and the practice of assessment seems to be exacerbated by the scaling of high-stakes assessments. Many statewide assessments, for example, seldom articulate the writing theory on which their assessments are based: “writing assessment practices still seem to be based on outdated conceptions of writing as a set of skills, mainly focusing on formal aspects of writing” (207).
Further, there did not appear to be much interact between measurement theory and writing theories. Alluding to Huot, there needs to be more interaction among measurement theorists and writing theories in order to “account for the complex nature of writing within diverse social and cultural contexts” (207). They advocate for an established field writing assessment theory that can have a hand in developing and validating assessment practices.
Joe's Prelim Notes 