To make use of the e-rater® technology, ETS offers writing assessment and scoring expertise to institutions that incorporate ETS scoring technology in their courses and online services. Institutions and companies may either use standard writing and instructor-created prompts or choose to develop their own.
If they use the standard prompts that follow ETS guidelines, the essay responses are submitted to the ETS grade-level scoring models and receive a holistic score.
If they choose to develop their own customized writing prompts, they must collect essay responses to these prompts, score them and transmit them to ETS so a scoring model can be built and certified. ETS will help institutions develop the prompts and score the responses so that a custom e-rater engine scoring model can be built by following the requirements outlined below.
Building a Custom Scoring Model
- ETS experts can assist with the development of customized essay topics that reflect national, state or district standards.
- ETS can provide expert advice as an institution develops its own customized topics, conducts scoring sessions and analyzes the writing score.
- Or ETS development specialists can do the development work and conduct the pretest scoring sessions, according to the organization's specifications.
- ETS development specialists can conduct essay scoring sessions and collect scoring data for customized model building while incorporating scoring methodology and quality assurance procedures.
- In addition to handling reader recruitment, training and monitoring, ETS offers an online scoring system that instructors and trained readers can use to score text-processed essay responses.
- When scoring is complete, the essay files and accompanying reader scores can be transmitted to the scoring engine for automated model building.
- Before building the scoring model for a topic, the e-rater technology will be trained by analyzing essays written on that topic.
- Essays must receive two scores from independent readers who have applied consistent and fair scoring standards.
- The pool of scored essays must also represent the entire scoring scale (for example, a scale from 1 to 6), so that the e-rater engine can be trained to recognize essays at each of the score points.
The e-rater Engine Models Requirements
- Scoring standards are clearly defined in a scoring guide
- An essay topic that reflects the writing task represented in the scoring guide
- A student population identical or at least similar to the population that will be using the topic
- Two independent reader scores for each essay response (readers must be trained to use the scoring guide)
- At least 350 student responses per topic in an electronic file, formatted according to ETS specifications
Contact us for more information and assistance in developing topics and guides, scoring essays and building customized writing prompts scored with your scoring guide.