Elliot, Norbert. On a Scale: A Social History of Writing Assessment in America (Chapters 5 & 6) (128 pages)
Chapter 5: Lone Wolves, 1966-2005
Elliot offers an overview of game-changing moments in writing assessment between the period of 1966-2005. Early in his history, he points to an overriding theme that describes these moments: “the era of protest had ushered a new sense of the individual and shown the doors to static conceptions of society. Organized protests would end, but the sensitivity toward the language of protest—the observation that testing encouraged destructive competition, for example, and that crimes against students could occur int hat environment—remained” (185). For Elliot, the language of protest began to appear in what we wanted writing assessment to emphasize: in other words, writing assessment began to emphasize the input of classroom teaching and de-emphasized the designs of the institutional models from ETS and College board.
He points, initially, to the work of Paul B. Diederich who “was emblematic of a new approach to writing assessment, one which provides strategies for classroom instructors to evaluate writing on their own without the help of assessment professionals in organizations such as ETS” (186). Again, we see a move to integrate the teaching of writing with the assessing of it. The re-focusing attention to the teaching and teachers of writing appeared to be a move toward validity. Diederich draws upon the work of John Dewey who claimed that descriptions we make from assessments must be understood as constructs. Writing assessment, thus, can only definitively claim what has occurred in the writing assessment, and our decisions and descriptions about its meaning should not be undertaken lightly. Elliot points to two demonstrating examples:
- When Diederich is diagnosed with cancer, he was told that the cancer had been detected and removed “early enough.” Somehow, such a statement from the doctor “puts the patient in the wrong if the operation does not come off, but there is very little he can do about it, since the tumor often gives no sign of its presence until anyone could call it ‘too late.’”
- Dewey writes of a child who sees a candle flame, “imagines it may be pleasurable to play with the flame, and thus, is burned and withdraws his hand. Yet this account assumes that the parts are prior to the whole; in fact, it is only the whole environment that matters” (189). In other words, there is no stimulus and response—a reflex arc that offers a simple narrative explanation. Rather, “there is no cause and effect, no stimulus and response; there are, instead, acts which are ‘strictly correlative and contemporaneous’” (189).
Such a view of assessment made way for arguments forwarded by some such asAlan C. Purves who was particularly concerned with disparities: cultural disparities represented in the assessments, disparities between curriculum and assessment, disparities in performance on the assessment due to language differences. “The ability to write a composition had more to do with cultural contexts than with a universal capacity for cognition, that there was a variation among nations regarding what was valued in composition” (200). Simply put, we have to look more widely at the contexts of interpretation: “there is no ‘best map’ of the world. There is no single rule. What is sauce for the goose is not sauce for the gander. One size does not fir all” (201). As Elliot writes, “culturally informed conclusions were, in the end, the brightest” (201).
Thus, the consensus view appeared to be that “not all tests should be abandoned, but that interpretations of results all be expressed within limits” (Lloyd-Jones qtd in Elliot 202). As such interpretations should, then (to return full circle to Diederich), include the voice of writing teachers—“in a forum that usually had been dominated by researchers and policy-makers” (203).
Under this paradigm, we can look more toward the validity of assessments themselves, and Elliot begins to point to some of the major assessment shifts in the last 50 years:
- Ed White, for instance, became instrumental in the change in writing assessments for the University of California System for admissions. “Changes in student populations, White had claimed, had rendered multiple-choice usage tests…not worthwhile testing devices” (207).
- Elbow and Belanoff, in SUNY Stony Brook, begin a portfolio assessment system that emphasized collaboration and negotiation: “Collaboration, they found, prompted teachers ‘to have to articulate to others (and thus for themselves), the basis for their judgments.’ And negotiation was the method by which the portfolios would be evaluated (i.e. through discussion)” (215).
However, Elliot also points to how, with the integration of assessment and instruction, we also must begin to attend to how we are teaching writing itself because classroom instruction has an impact on performance on an assessment. In other words, there is no cause-effect or reflex arc of teaching to assessing, but a circle. Per Odell, “if there is a failure of instruction, there is little that assessment can reveal…it is hard to have meaningful assessment unless there has been good teaching” (Odell qtd in Elliot 219). Elliot demonstrate Odells point, “Even advanced students are often taught to write to a formula, [Odell] had concluded, and assessment often reward sosely those rote formulas. Such formulaic instruction has little epistemic or rhetorical value and remains uninformed and superficial. ‘What might be the floor becomes the ceiling’” (220).
Elliot closes by discussing the emergence of the E-rater and the history of the SAT II (with text-based writing). With the SAT II—particularly in the structure of the prompts to be open-ended—the emphasis again was on how such assessments might be suited to aid in an informed educational process. However, he alludes, very briefly, to a kind of open question:
To Richard C. Atkinson’s [the UC system president who was instrumental in the SAT II change] granddaughter, none of [the circle of teaching and learning] mattered. She would be among the firs to take the new SAT. She was not hesitant to accuse her grandfather of complicating her future. She had liked the old SAT. (242).