Adler-Kassner, Linda, and Peggy O’Neill. Reframing Writing Assessment to Improve Teaching and Learning(Chapters 1, 2, 3) (80 pages)
Chapter 1: Higher Education, Framing, and Writing Assessment
As Adler-Kassner and O’Neill explain, writing appears to be everyone’s business; “everyone, however, doesn’t agree on how to define ‘writing,’ let alone ‘good writing’ or ‘writing ability’” (8). How, then, if writing is everyone’s (from the classroom up to federal government) business, do educators and researchers interact with these constituents to design writing assessments that are fair? The purpose of the book, as they write, is to situate instructors and program directors at the center of discussions and designs of writing assessments.
Chapter 2: Framing (and) American Education
Frame acts as the organizing principle of Adler-Kassner and O’Neills book. Similar to Burke’s terministic screen, a frame is “a perspective that shapes understandings of situations or circumstances…they shape individual’s or group’s perceptions of what is and is not plausible/in the picture/focused/visible…language and terminology affect the nature of our observations and understandings of reality” (15-6). A frame draws its strength from the harmony among the stories that shape the frame itself; however, creating harmony—or connections—among stories often occludes other possibilities or other stories that could destabilize the frame. Accordingly, it is often easier to see frames “and the process through which they have been constructed when the structures they support become destabilized” (18). Accordingly, frames become normalized into “common sense,” enacting a hegemonic or dominant ideology that constitutes one’s natural reality.
The authors then offer the ways in which the U.S. has framed formal education in a couple of ways. First, they outline how some have framed education—early on—through it’s function to foster productive citizenship. They point to two initial senses of the term:
- First, “productive citizens are those who contribute to the growth of the democracy, and that all citizens have the capacity to make this contribution. School…should [tap] into [students’] natural interest and [help] them understand the connections between those interests and the nation’s broader goals” (20-1).
- Second, “productive citizens are those who fulfill particular roles identified for them (by managers), and these roles are predetermined to contribute to the nation’s progress in particular ways. In school, students’ abilities are identified and then learners can be sorted into different groups in order to have these abilities cultivated appropriately” (21).
More specifically, such a distinction can be re-aligned through public good or private good. Education serves public good when education prepares students to take on the full responsibilities of citizenship and their economic roles in a competent manner. But education also serves a private good as a commodity: “the only purpose of which is to provide individual students with a competitive advantage in the struggle for desirable social positions” (21). Under this frame, assessment takes on a few roles, namely, to promote student accountability to ensure they are meeting the needs of the state, but also for purposes of ranking for private good.
In another frame, the focus is more economic, driven more by supporting the individual within the system rather than the system itself: this means that while more students might have support to attend college, the systemic and structural issues associated with colleges (from inadequate instructional staff, to understaffed support programs, to deferred maintenance on facilities) would not be addressed” (24). In terms of assessment, such a frame remains widespread as the individual student is assessed to indicate the collective success of the system or institution.
Such frames have developed a mistrust of educators: “postsecondary educators no longer understand the shape of the democracy. Thus, the report says, educators and institutions must either retool, or run the risk of surrendering agency that enables them to prepare students through education” (26). In other words, there is a move for standardization whereby such standards will be set not by the institutions, but by federal officials/agencies. As of now, “accrediting agencies do not set standards; instead, accreditors ensure that institutions are setting standards, assessing how and whether those standards are being achieved, and constantly working to improve teaching and learning within those standards” (28). Moves such as these indicate a frame of accountability and the creation of accountability systems that seek to standardize assessments thus ensuring that “institutions are achieving these standards, thereby circumventing the authority of disciplinary experts and undermining the peer review system upon which accreditation (and, in fact, the entire academy has been based” (31).
However, the authors maintain that postsecondary educators who are experts in their fields, “understand the kinds of citizenship to which their disciplines are best suited, and how the knowledge of their disciplines should be communicated to students in order to suit students for those citizenly roles” (31). Thus, echoing much of the same refrains from Huot, assessments should remain and reflect the disciplinary expertise of those practicing in those disciplines: assessments should be directed internally to improve the work of their classroom or programs. However, as assessments are scaled up for general public good, the emphasis is on proving “what is (or is not) taking place in the classroom” (39).
Chapter 3: The Framing of Composition and Writing Assessment
Similar to the pervious section looking at the education system writ-large, the authors look at the ways in which composition and writing assessment have been historically framed. They allude to Harvard’s written admissions exam as a landmark for a shift toward composition’s frame within the “emerging industrial economy,” and how writing assessment became a means of ranking and measuring whether students were able to meet the needs of the industrial economy. Alongside this frame, there was also a technocratic (or administrative) frame that “privileged management, efficiency, and tracking in education, as in industry and elsewhere” (45). “a mechanistic paradigm that studied language by reducing it to discrete behaviors and that defined language growth as the accretion of these particular” (45). The focus on mechanics and error stressed, of course, the quantification of writing; thus, composition generally could be taught by contingent faculty—low-level, entry-level workers—who would be able to draw attention to surface level concerns rather than the more complex work of writing. In this way, teacher’s judgments were devalued and—along with overtures of efficiency and objectivity—there was an embrace of standardization in the assessment of writing.
However, the authors also note a paradox in the academy: there appeared to be an effort to “define the work of members of the academy, especially their methods and knowledge bases, as something that could be understood only with particular professional training and that could be itself judged—assessed—only by peers who shared this specialization” (51). In other words, academy began to see the specialization of particular fields of study. However, at the same time, psychometricians and educational measurement experts became the de facto authorities on acceptable assessment theories and practices across all fields of work, including writing.
However, there were others like Fred Newton Scott at the University of Michican in the early 1900s who framed language as “social, contextual, and grounded in community. Language had the power to change individuals and to effect change in broader contexts” (54). Thus, while he was not much appreciated during his time, his rhetorical frame for composition re-emerged in the 1960s. The rhetorical frame “acknowledges that writing is a form of communication governed by the writer’s purpose, the message to be delivered, the audience to be addressed, and the context surrounding the writer, audience, and text” (56). There was a focus on the meaning of the text: a text cannot be understood outside of its context. The effectiveness of a text, accordingly, needs to be “determined in light of the particular situation. A text can be transported to other situations, but its effectiveness and, in fact, its meaning, can only be determined in context” (56-7). While students cannot be taught to master every situation, they can be taught a critical metalanguage to analyze situations and determine needs.
The authors also point to a sociolinguistic and literacy frame, one that emphasizes the social nature of language and how community is integral to expectations and discourse. But more, it is not only that language derives meaning from context (beyond linguistic codes), but language, itself, shapes contexts and interactions.
As these latter frames indicate, context needs to be a significant component in the judging of a text. Standardization, then, appears counterintuitive to the evaluation of writing since it is not likely that there can be generic qualities that apply across all situations; instead, criteria should be designed around specific contextual situations: “according to contemporary composition theory, there is no universal definition of ‘good writing.’ Rather, good writing needs to be defined by what is appropriate and effective for a particular purpose, and a particular context” (64).
Adler-Lassner and O’Neill turn their attention to some practices of writing assessment—in/direct measures and portfolio—and much like others, make the claim that these methods, in themselves, are not valid or invalid, but rather validation is determined by the context of use. For example, portfolios have sometimes been scaled up for higher large-scale purposes that have essentially reduced it within psychometric frames.