Whithaus, Carl. (2005). Teaching and evaluating writing in the age of computers and high-stakes testing. Mahwah, NJ: Erlbaum.
Whithaus begins to outline the key ideas that will frame his book. At it’s heart, Whithaus’ exigence centers on the emergent focus on the multimodal and technological aspects of composition—i.e. multiliteracies and multimedia. For Whithaus, it would not necessarily be prudent to frame the focus on multiliteracies as only applicable to a new understanding of print based writing situations; rather, multiliteracies prompts us to question fundamental aspects of how we teach and assess writing: “what is the point of teaching writing or composing skills? What knowledge or sills do educators aim to produce? What ideas do they want students to come into contact with in language arts, English, and composition stud nets? And why do they want them to engage with these ideas?” (xx) A participation framework, for instance, begins to help us consider these questions in relation to multiliteracies. It refers to the ways that the speaker/rhetor evokes different modes of audience participation: “effective communication is a dialogic process situated within a social context” (xx). Fundamental in this framework is how the rhetor grapples with multiple modes or media of participation and interaction. For our students, they can create multimodal compositions that draw upon “graphic textual and oral communication formats and multimedia composition uses IT to present these various features” (xxvi). He continues, the choices of medium or mode require “the student to consider their audiences, organize their information for that audience, and adapt—or design—the presentation of that information for that audience, and particular medium and a particular social context” (xxvi).
For the teaching and assessing of writing, students must be taught to contend with multimodal formats which thus involves evaluating the complex compositions they produce. As such, because these activities are situated, it is imperative that—as teachers and students—we are forthcoming in describing and articulating the ways in which these practices are situated. “An authentic evaluation should be able to assess student compositions, both multimedia and print, by including as many situational elements as possible. Because communication always happens in context, the most accurate representations of students’ writing and composing are not granted form assessments that attempt to capture decontextualized skills” (xxvii). Key to his assessment model is descriptive evaluation: description of process allows for a record of the kinds of situational elements involved in a multimodal composition and allows students to articulate what is typically tacit.
Chapter 2: Writing (about) Sounds, Drawing Videos: Multimedia Compositions and Electronic Portfolios
Whithaus begins to note that the writing studies in the academy—particularly the humanities—has been largely separated from the public both through the “conceptual apparatus and the discourse of the academy” and, more important, the medium in which we work, the printed word. From Greg Ullmer: “it is time for the humanities disciplines to establish our cognitive jurisdiction over the communication revolution” (19). Simply put, the academy needs to begin to understand the very multimodal ways in which writing occurs—by ignoring such aspects of writing waives our ability to stake a claim in the study of writing itself. As such, Whithaus looks to genres of composing that often do not receive attention in the academy such as rap or hip hop.
In order for Whithaus to explain how we can assess such a foreign text (to the academy), he looks toward how we can understand its situatedness: “to judge a rap song as a composition, one must understand the situation in which it was created. This act of evaluation include snot only understanding the genre of the song but also the context—the social situation—within which the piece was composed…This may mean turning to rich interactions between genre or discourse conventions and material conditions” (20). Central to Whithaus’ concern in approaching the assessment of the text is when a student says “You just don’t get it.” To “get it” implies an assertion about understanding and speaking to the context in which a composition is created. Looking systematically at “getting it”: “Once the audience ‘gets it,’ they are into the ‘context of knowledge.’ In fact, they are part of creating a context of knowledge that values the particular communicative practices championed within a discourse community” (21). Turning to teachers, it is imperative that teachers, to assess such work (to ‘get it’), they “must understand the genre in which the students are writing and the social situation(s) within which the genre works” (22).
Whithaus points to student examples from Sirc who discusses a collaboratively made list of terms in hip hop circles. Such glossaries—or what Sirc class inscribing, cataloguing words, ideas, material that might become useful for the next writer—begin to offer teachers, parents, etc. a context to understand the texts.
In another example, he looks at—what can only be described as—a giant “fuck you” essay to David Bartholomae: a student decides to shrug off the writing assignment and disrupt the expectations itself. Bartholomae, admitting that it was both a failure in terms of meeting expectations and the only memorable piece he has had in the en tire class, does not have a context in which to understand or “get” the piece itself. Whithaus connects such a piece to Shaughnessy’s attention to error: “errors that aren’t errors, but appear to be when looked at through the logic of standardized English” (26). Whithaus remains skeptical of this particular example, itself, (himself admitting that he also doesn’t have a context to understand it), but is nonetheless able to use this dissonance to question how we might go about assessing such work.
For him, the question of assessing the text lies upon (1) questions of accessibility of this and similar kinds of discourse, (2) questions of relevance of these discourses; (3) and situations in which these discourses are valued. But what tethers each of these questions of assessment is the position, the situatedness of assessment: “evaluation is always already situated, but the situatedness does not equate to an inability to judge and apply criteria…Without a situation within which to be made, the very idea of value becomes meaningless” (26). For assessment, “it is necessary to become better at articulating the importance of situation to writing assessment and evaluation, or perhaps not simply articulating the importance but rather designing assessment systems where situation is incorporated into the evaluation” (27).
However, again approaching the assessment of the examples from Sirc and Bartholomae, Whithaus again admits that he cannot pretend to have the situational knowledge—or domain knowledge—necessary to evaluate such texts. He does, however, see potential in assessing these texts if students were to curate and stage other adjacent texts that would contribute to a better contextual space to evaluate the texts (see: Gallagher).
Chapter 3: Situation(s): Using Descriptive Evaluation
Here, Whithaus begins to look more closely at the techniques of an effective assessment system:
- Interacting with students as communicators, pointing to how their text participates within social contexts (rather than simply teacher expectations).
- Describing the processes and products of student learning rather than enumerating deficits.
- Situating pedagogy and evaluation within systems that incorporate rather than exclude local variables; and
- Distributing assessment among diverse audiences, including student peers.
Whithaus offers a kind of overview of these techniques and how they are enacted in response to student writing. He is particularly interested in a method of descriptive evaluation. As an analogy, he discusses such a method as a kind of pictures from a camera by the student and the teacher: the teacher takes a picture of the work as well as the student. Between these descriptions of reality—which are, of course, selections and deflections of reality (per Burke)—a teacher can determine a grade. But Whithaus further notes that distributing such descriptions across a body of others—including student peers—can further validate this means of evaluation. A distributed descriptive method provides an opportunity to situate the composition within multiple angles from which the work can be described.
Although he does not contend with the question of distribution, he looks closely at whether the student “gets it” in terms of the discourse conventions embedded in the social activities of different academic fields: the technique of description, for instance, acknowledges both how the text interacts within a social context as well as how it is situated.
The descriptive method “begins by simply restating what the finished compostiions are…the descriptive evaluation also notes the potential another text to emerge from the assignment” (37). In other words, these comments “I simply state what the student produced” (37). Further the descriptive method also focuses on how the text participates within a wider social context, i.e. the description will comment upon the interactive qualities of text. He points out that, oftentimes, teachers’ descriptions of interactive qualities are contained within the features of interaction with the teacher: “these ‘interactive’ directions drive the essay away form its relevance for the student and make it relevant for the teacher (what-i-would-have-liked). Descriptive evaluation is not about what-I-would-have-liked, but rather about what is” (38). How does the text interact with its social context, its audience? It’s also important to include student quotations from students’ compositions as a means to help students resee what they have composed.
Overall, “a composition needs to be judged for how well it works in a given situation” (42), and describing the compositional features and how it functions within context begins to emphasize this goal.
Chapter 4: Negotiating Assessment and Distributive Evaluation
Here, Whithaus looks more closely at what a distributed evaluation method would look like. A participatory framework “suggests multiple agents are evaluating a speaker. When a child on the streets of Philadelphia speaks and that speech is inflected for an audience, and opens to invite members of that audience into the debate—to take sides—then the audience member sare always evaluating and creating the meaning of the participatory framework…this multiple interactivity urges moving beyond a student-teacher vision of the composing process and to see the student within a framework of other students, the teacher, and potential audience members outside of school” (49). We might think of Bakhtin and how discourse involves the participation of others, meaning deriving from the ways we participate with texts and the contexts of their usage.
Toward the classroom, Whithaus sees negotiated evaluation methods—collaboratively developed evaluation criteria—appear to align with the goals of a distributed evaluation method. In allowing for such evaluation methods, Whithaus “hoped to provide the students with some control over the curriculum and the ways in which their learning was evaluated” (52). However, he also notes, “I want to share control of the course with students, but I also need to demonstrate the seriousness and validity of discussing writing and my expertise to facilitate these discussions” (54). In other words, it is fine to invite students to participate in the evaluation of the class, but he is also aware that it is his duty to ensure students remain within some kind of appropriate scope/focus.
He concludes, “students will bring preconceived notions about what English teachers value in writing and what should be marked as good, correct, and effective, but an open discussion airs out these assumptions and help students and teachers begin to consider what they value in writing, and more narrowly—and perhaps more effectively—articulate what they will value in each other’s writing over the course of the next 16 weeks” (65). In terms of distributed nature of descriptive evaluation methods, such articulates of values will aid in how students distribute their evaluation of compositions.
Chapter 7: High-Stakes Testing and 21st Century Literacies
Whithaus looks to the impact on high-stakes tests, such as those implemented under No Child Left Behind, on how we think about the teaching of writing. One of the key features of the No Child Left Behind legislature is the idea of stronger accountability: “Holding administrators and teachers responsible for student performance requires a system of determining effective performances—in short, it requires testing” (104). The focus on the accountability system as positioned assessments uniquely in relation to teaching. For instance, it becomes commonplace to believe that what is tested is what is valued; thus, “students, parents, and teachers come to believe that if something is not tested, it is probably not very important to learn” (105). However, as Whithaus points out, it is often the case that such tests focus exclusive attention to low-level skills and facts regarding writing. This places teachers in an ethical dilemma: “yet [teachers] are obligated…to prepare students to meet the academic and real-world communication situations they are going to encounter, as well as to prepare them for whatever standards-based test is in place at the moment” (105). Whithaus, however, is relatively optimistic about how teachers handle this dilemma: “what surprises me over an dover again, is the resilience of teachers and their sense that given time they will both come to understand what, if anything, a test can add to their curriculum and what parts of their teaching are too valuable to give up to test prep time” (105). To put another way, teachers needs to be able to make judgments about where the content of test can aid in curriculum and where it doesn’t.
But with high-stakes testing, it is often quite easy to show improvement in test performance, but such improvement may only indicate that teachers are better able to adapt students to succeed on the test rather than demonstrate the student’s writing proficiency. However, Whithaus, for a moment, concedes that with good tests, such an assumption about writing proficiency can be helpful—teaching to the test, then, doesn’t seem to bad if the test is valid (see; Wiggins). However, he also notes that under such an assumption, “writing teachers are losing control of the curriculum” (108).
And again, these tests seem to be only focusing on print-based literacies—it is imperative that teachers include attention to multiliteracies in the curriculum, even if it isn’t tested on high-stakes tests. Teachers, in other words, cannot wait for high-stakes testing to catch up to current scholarship in 21st century literacies.
Chapter 8: Tools (AES) and Media (blogs)
In this chapter, Whithaus’ central concern is the way we integrate print-based literacies with multiliteracies. He frames such discussion with the work of Brandt and her concept of residual literacy: “the skills and strategies acquired by one generation and passed on to the next, despite changes in public discourse conventions and technology” (116). There are many systems in place in education that seek to preserve the logics of old media; however, Whithaus (again) appears optimistic about the ability to change and adapt. New writers of the 21st century are able to compromise; however, it’s important to take note of the technologies and media in place that invite particular kinds of knowing; however, we must acknowledge how writers remain agentive in the way they write. His central question is how to integrate the residual forms of print-based writing and the rapidly accumulating new forms of multimedia composing” (118).
He looks both at AES (such as e-raters) and blogs (as networked, distributed spaces of interactions). His most salient point is that AES is, in some ways, from the same family as the word processor: where AES evaluates writing with one particular situation (and does not often reward writing within another situation), they can potentially be helpful much like how the word processor is used by students in their writing process, making decisions based on the feedback from the software.