Holistic Scoring: Method

The Writing Assessment Program uses a holistic scoring method to score the all exams and portfolios.  Elliot, Plata, & Zelhart offer the following definition of holistic assessment:

To view a sample of writing holistically is to attempt to view the writing as more than the mere sum of its elementary parts.  In considering a sample of writing from a holistic perspective, readers do not judge separately the singular factors--treatment of topic, selection of rhetorical methods, word choice, grammar and mechanics--that constitute a piece of writing.  Rather, raters are asked to consider these factors as elements that work together to make a total impression on the reader.  It is this total impression that is sought in holistic scoring. (17)

In holistic scoring, essays are not given low scores just because they contain many mechanical errors, nor are they given high scores just because they are well organized.  The reader considers the overall impression created by the student's writing sample and assigns a score consistent with that overall impression. 

The score a reader assigns is defined by a set of criteria that define important traits in all writing: at Southeast, the criteria include focus, organization, development, style, correctness, and if necessary, references.  These criteria, in turn, define the scoring scale applied to the essay; Southeast's scale runs from 1 (incoherent) to 6 (clearly excellent).

The evaluation scale is divided into two halves: the top half is labeled "mastery," and the bottom half is labeled "non-mastery."  The three scores in each of the halves then represent high, medium, or low levels of mastery or non-mastery.  Bearing in mind all of the writing traits to be considered, a reader first decides whether the essay shows, overall, some kind of mastery or some kind of non-mastery.  This is a yes or no decision.  Then, having made that decision, the evaluator decides whether the level of mastery or non-mastery is high, medium, or low.  If the reader decides that the test shows low mastery (or just barely "yes"), he or she gives it an overall score of 4.  For medium mastery, the score is 5, and for high mastery the score is 6.  In the same way, essays demonstrating high non-mastery (or just barely "no") are assigned a 3.  For medium non-mastery the score is 2, and for low non-mastery the score is 1.

Each essay is initially scored by two trained readers in a double-blind process: that is, the two readers do not know each other's score or each other's identity, nor do they know who wrote the essay.  If the two readers give the essay "matching" or "touching" scores, the essay's score is figured from the first two scores.  Otherwise, in the case of "discrepant scores," a third reader scores the essay; the essay's final score is determined by a combination of the three scores.  Essay scores are determined as follows:

Work Cited

Elliot, N., M. Plata, & P. Zelhart. A Program Development Handbook for the Holistic Assessment of Writing.  Lanham, MD: UP of America, 1990.