White, Edward M. “Holisticism” AW
White makes a distinction between holistically scoring writing and the “analytic spirit” that tends to “break things down into constituent parts.” As he writes, “to proceed holistically is to see things as units, as complete, as wholes” (19). Holisticism—a direct measure of writing—is placed in opposition to both multiple-choice testing and analytic approaches to writing.
White is very conscious of the consequences of choosing certain assessments of writing over others. As he writes, “Testing shapes curriculum,” thus, “multiple-choice usage tests were leading some teachers to doubt the importance of writing in the schools as well as defeating those who were teaching literature and creativity” (20). He notes that multiple-choice test were a means of achieving accurate reliability—the almost fetishized concept in testing culture—it’s question of validity is questionable. However, he also understands that direct measurements of writing like holistically scoring can only be accepted component of writing testing if it met two criteria: reliability and economy (or cost-effectiveness). Both of which must be met “without losing face validity as a legitimate test of writing skill” (21). He defines holisticism within the frame of these two criteria.
First, he dispels the confluence of general impression scoring and holistic scoring—general impression is not often guided by any systematic procedure and “a paper’s score depended on the accident of who wound up as reader, rather than on its quality” (22). Such a means of scoring does not meet standards of reliability. Rather, holistic scoring involves six procedures and practices needed for reliability:
- Controlled Essay Reading: scorers should develop a positive social situation, developing a sense of community or interpretive community “that determines and enforces the standards of measurement” (23).
- Scoring Criteria Guide or Rubric: establish a rubric that “makes a distinction between papers on a continuum from the worst to the best” (23).
- Sample papers: “Not until all readers are in close agreement on the scores of these sample papers, and on what characteristics have determine the score, can a reliable reading begin” (23). Scoring sample papers together, again, becomes a means of fostering a community, one where members agree upon common standards and thus internalize the scoring scale.
- Checks on the Reading in Progress
- Multiple Independent Scoring: “a one-point difference is allowable; two-point differences are read a third time in order to resolve the discrepancy” (24).
- Evaluation and Record Keeping
However, White also notes that holisticism has the capacity to be misused and abused. For example, White notes that while we may assume a score derived from holistic scoring reliable, it doesn’t tell us much about the student: “all we have is a single score, where we might wish to have a profile” (25).
In conclusion, White says that the holistic scoring, in fact, has the benefit of influencing how educators understand how to assess writing: “after many hours of scoring papers to intelligent criteria, seeing student writing holistically, it is not easy to return to the idiosyncratic, arbitrary, and mechanical response to student work that is standard practice for many of those teaching writing classes” (27). White also sees the value of developing the a sense of community as a key component to the trickle-down effect of holistic scoring: scorers should feel valued and comfortable and not coerced during calibration.