George Hillocks, Jr., in his essay, “How State Assessments Lead to Vacuous Thinking and Writing, ” argues that in many cases, high-stakes testing provides students with an impoverished context for writing, and therefore rewards surface, mostly evidence-free writing. Instead, students are indirectly encouraged to supply claims and warrants instead of evidence. In Aristotelian terms, we might say that students are using intrinsic proofs (invented by language) instead of extrinsic proofs (existing outside of the writer’s mind).
The solution, Hillocks claims, is to enrich the writing contexts for high-stakes testing, and this idea is reinforced by Arthur Applebee, in his use of the framework for the 2011 National Assessment for Educational Progress in writing. The section, “The Issue: What Should Students Write About?” references Hillocks, and in the outcomes section, notes:
Although many use items very similar to those in NAEP, others, such as New York State, base writing on extended reading passages, or include at least some classroom-based writing as part of the assessment (Kentucky, Vermont). A more general issue for assessment developers is whether it would be useful to increase the content load of student writing prompts, and if so, how this could be done within current assessment frameworks or through extensions of them (p. 91-92, emphasis mine)
It is difficult for me, quite honestly, to imagine an objection to this practice, at least in terms of validity. I assume we can all agree that if we are requesting writing samples from students, we want thoughtful, well-reasoned work–work that would stand up in contexts outside of a writing assessment. A greater amount of context, it seems to me, would enrich student writing. Real data could be provided, and various viewpoints on an issue could be shown. Students, in this way, would be able to use real, detailed data, instead of generalizations.
So what’s the problem? Applebee writes, “One possibility, particularly if writing and other assessments become computerized, would be through the adoption of some common metrics for assessing quality of writing across assessments in different content areas” (p.92). This seems to address the problem of students writing in different areas of prior knowledge–for instance, if one student is writing about the Emancipation Proclamation, and another is writing about tree frogs.
But the idea that Hillocks advances, as I read it, is not that students be tested on what they already know, but that we enrich prompts, providing more information about the writing context. This would not seem to be a problem from a reliability standpoint, because we could give all students of a given level the same prompt, and we could assess it with the same constructs with which we already assess writing samples. Perhaps a bit more time would need to be provided, but otherwise, the task could stay more or less the same.
Les Perelman, in “Information Illiteracy and Mass Market Writing Assessments,” however takes the practice of enriching writing prompts firmly to task, claiming that this sort of condensed data do not constitute “real arguments,” and that students are encouraged to use information improperly (133). The student has to accept the authority of the information without checking it.
Perhaps this rather damning summary is fair, and perhaps Perelman is right in suggesting that summative assessments should resemble literature reviews more than arguments. Nevertheless, while high-stakes testing remains more or less in its traditional form, I can see few reasons not to enrich writing prompts.