Verbal Reasoning and Quantitative Reasoning Measures
The Verbal Reasoning and Quantitative Reasoning measures are section-level adaptive. This means the computer selects the second operational section of a measure based on the performance on the first section. Within each section, all questions contribute equally to the final score. For each of the two measures, a raw score is computed. The raw score is the number of questions answered correctly.
The raw score is converted to a scaled score through a process known as equating. The equating process accounts for minor variations in difficulty among the different test editions as well as the differences in difficulty introduced by the section-level adaptation. Thus, a given scaled score for a particular measure reflects the same level of performance regardless of which second section was selected and when the test was taken.
Analytical Writing Measure
For the Analytical Writing section, each essay receives a score from at least one trained rater, using a six-point holistic scale. In holistic scoring, raters are trained to assign scores on the basis of the overall quality of an essay in response to the assigned task. The essay score is then scored by the e-rater® engine, a computerized program developed by ETS that is capable of identifying essay features related to writing proficiency. If the human and e-rater engine scores closely agree, the average of the two scores is used as the final score. If they disagree, a second human score is obtained, and the final score is the average of the two human scores.
The final scores on the two essays are then averaged and rounded to the nearest half-point interval on the 0–6 score scale. A single score is reported for the Analytical Writing measure. The primary emphasis in scoring the Analytical Writing section is on critical thinking and analytical writing skills rather than on grammar and mechanics. (Read the "Issue" and "Argument" scoring guides and the Analytical Writing Score Level Descriptions.)
During the scoring process, test takers' essay responses on the Analytical Writing section are reviewed by ETS essay-similarity-detection software and by experienced essay raters. Learn about invalid scores.
Monitoring Rater Performance on the Analytical Writing Section
Many different strategies are used to ensure that all raters use the same scoring standard. At the beginning of each scoring session, raters must score a calibration set of 10 previously scored essays with 90 percent accuracy before being permitted to score operational essays.
To maintain quality during operational scoring, previously scored essay responses (validity responses) are interspersed among unscored operational essays to monitor each rater's scoring accuracy.
During operational scoring, previously scored essays (monitor essays) are interspersed among unscored operational essays to monitor each rater's scoring accuracy; raters cannot distinguish between the two kinds of essays.
Scoring leaders (very experienced raters) also monitor raters' performance throughout the scoring session by reviewing raters' scores on operational essays, monitor essays and calibration essays and by monitoring score distributions. Scoring leaders also provide raters with ongoing support and guidance. Raters who deviate from the acceptable level of accuracy are retrained or dismissed.
Are test takers who use alternative ways of developing an argument scored fairly?
Test takers may use any one of a variety of strategies to structure their essays. Readers are explicitly trained to accept any strategy in an essay that meets the essential requirements of the essay task, that is, a response that provides the information required by the essay prompt.