Video Title: Major Field Tests Flash Tour 2013

People in this video

Bill Wynne – Narrator

Introduction [1:47]

[music playing]

On-screen: [Animated ETS Major Field Tests LOGO]

Bill: What if you could consistently evaluate students' mastery of their major field of study?  And what if you could use that data to prove – and improve – the effectiveness of your programs? With the ETS Major Field Tests, you can.

On-screen: PROVE and IMPROVE – the effectiveness of your programs

Bill: The ETS Major Field Tests are comprehensive outcomes assessments offered for more than a dozen undergraduate fields of study and for the MBA. The assessments measure the critical knowledge and understanding obtained by students in their major.

On-screen: 

Bill: Major Field Tests can help your institution track and evaluate how well your students have mastered their field of study; enhance teaching and learning with reliable, actionable data; demonstrate the strengths of your program to prospective students, faculty and stakeholders; gauge your institution's performance for accreditation and accountability; and gain access to comparable data for 1,500 institutions nationwide.

On-screen: 

Bill: ETS has a rigorous process to ensure that every question on every test undergoes statistical analysis, yields the expected result, and contributes meaningfully to the test.  Input from faculty members, who are experts in their field, contributes to the proven validity and relevance of each Major Field Test, providing you more confidence and tangible value in your results.

On-screen: 

Bill: ETS has 60+ years of experience in creating high-quality, research-based assessments. 

On-screen:

Bill: ETS develops the Major Field Tests using the highest professional standards, which are based on AERA and NCME standards, to ensure the results are fair, valid and reliable.

On-screen: Major Field Tests

Bill: Prove and improve the effectiveness of your programs with the ETS® Major Field Tests. 

On-screen: PROVE and IMPROVE the effectiveness of your programs

Benefits & Features [2:45]

Bill: ETS Major Field Tests measure critical knowledge and understanding obtained by students in their major field of study. The tests are available in both paper-and-pencil and online versions, and deliver unparalleled access to actionable data. The proven validity of the Major Field Tests can help you support continuous improvement, and meet accreditation requirements. 

On-screen: [left-hand navigation links]

Bill: Major Field Tests go beyond the measurement of factual knowledge. They help you evaluate students' ability to analyze and solve problems, understand relationships, and interpret material from their majors.

On-screen: Go beyond measurement of factual knowledge

Bill: Comparative data from over 235,000 students across nearly 4,000 programs nationwide help you with external benchmarking efforts for both regional and program accreditation. Users can select among participating institutions to create their own peer reference group. 

On-screen: 

Bill: The test delivers actionable score reports you can use for curriculum improvement. Subscores and assessment indicators help you identify potential areas of need. You can conduct even more detailed curricular analysis and give your faculty even more control with a question-by-question report of your students' performance.

On-screen: 

Bill: A centralized web portal provides you with access to your data and reports, online, anytime. 

On-screen: Centralized web-reporting portal provides access to your data and reports, online, anytime

Bill: Scores for online versions are available immediately after your students complete the test. Scores for paper-and-pencil versions are available within a few days of receipt of your answer sheets by ETS. 

On-screen: 

Bill: Both versions allow you to meet your own scheduling and staffing needs. The online version provides added convenience and flexibility. There are no test books to securely store, no answer sheets to return, and you can never run out of inventory. And, you no longer need to spend valuable class time administering the test. Students can take the Major Field Tests in any proctored computer lab on campus. 

On-screen: Added convenience and flexibility with the online version:

Bill: Plus, coming in 2013, administering the Major Field Tests will be even easier with proctored, off-campus test sessions. Locally administered proctoring remains available, but the off-campus option adds greater convenience, security, no loss of valuable classroom time, and saves your institution the cost of finding a suitable test administration space.

On-screen: Remote proctored test sessions

Bill: Online and paper-and-pencil testing, easy to access actionable data which can be used for a variety of purposes, and flexible administration options – the Major Field Tests provide you with the options and data you need to make your assessment process efficient, flexible and powerful.

On-screen: Efficient, flexible and powerful

Test Design [4:42]

Bill: The ETS Major Field Tests are designed to the highest quality standards, based on rigorous models including ones created by the AERA and NCME. But how does ETS create an assessment? How do you build validity, reliability, and fairness?

On-screen: [left-hand navigation links]

Bill: The Major Field Test development process takes up to 18 months, even for test titles that existed previously. Titles are refreshed periodically to ensure that they are always in step with changes in your discipline and how you are teaching it. Even when a test already exists, the entire test development process is undertaken for every new version of a test. There are no shortcuts. 

On-screen: [Screenshot of lifecycle chart]

Bill: This means that ETS is in a constant cycle of test development for the Major Field Tests.

On-screen: Constant cycle of test development for the Major Field Tests

Bill: For each Major Field Test, ETS employs a variety of content experts – former and current faculty in their respective disciplines. The same experts also lead the development of GRE®, Praxis, and other tests. 

On-screen: Content Experts – former and current faculty

Bill: The test development process begins with a curriculum survey. Hundreds of programs across the country participate, both those that use Major Field Tests and those that do not. This scientifically-validated, qualitative study generates data about how a discipline is being taught right now. The results help ETS content specialists understand which concepts experts in the field believe are essential to students' success in their program.

On-screen: Curriculum survey

Bill: ETS then brings together a Test Development Committee, a panel of experts in their discipline. It is comprised of faculty and chairs from a variety of programs – large and small, private and public, urban and rural, HBCUs and HSIs. The committee convenes at ETS to help the assessment development team analyze the results of the curriculum survey. 

On-screen: Test Development Committee

Bill: This analysis results in a test blueprint, or test description document. The blueprint indicates which specific content should be represented on the test, and which specific concepts students should have mastered. The test blueprint for the discipline of interest to you can be viewed on the Major Field Tests website.

On-screen: Test Blueprint

Bill: The test blueprint determines what kinds of items the test should include. When asked who writes the questions, ETS often responds, "You do!" ETS trains faculty who are currently teaching at institutions across the country to craft questions that match the blueprint and meet ETS standards of fairness. This ensures that the questions are fair, current and relevant to your discipline. If you have never written questions for an ETS Major Field Test, a colleague you admire and respect likely has.

On-screen: Item Writing

Bill: Before a test is administered, it must undergo the ETS fairness review. An external panel of experts in fairness issues conducts a rigorous review to ensure that the test is as unbiased as possible. Questions deemed to advantage any group over another are removed. 

On-screen: ETS fairness review process

Bill: The process doesn't end there. The test is then administered to students at a representative sample of institutions – often called a "norming population." ETS uses the data from these administrations to ensure that the test works the way it is expected to. 

Although a qualitative fairness review has already been completed, these data can be used for statistical analyses such as Differential Item Functioning to ensure that questions yield the expected results across all groups of students. Item difficulty is confirmed, and questions are validated to ensure that each has only one correct answer. This same data is used to make scores on a new version of a test comparable to scores on previous versions of tests in the same discipline. 

This statistical equating process allows your program to make comparisons across different groups of students over time – even if you have administered several different versions of the test. 

On-screen: Statistical Equating

Bill: Only then can ETS report official scores on the new version of the Major Field Test in that discipline.

On-screen: Official scores

Bill: Curriculum surveys; external content experts; questions written by you and your colleagues; qualitative and quantitative fairness reviews – this is the validity chain, the aspects of responsible test design and development that ensure validity, reliability, and fairness. This process provides you with confidence and tangible value, and delivers the analytical power for which the ETS Major Field Tests are renowned.

On-screen: Validity chain

Score Reports [4:48]

On-screen: [left-hand navigation links]:

Bill: A test designed to rigorous quality standards means powerful results and data you can actually use. The ETS Major Field Tests go beyond yielding a single score. They also provide multiple score types which you can use for a variety of purposes to contribute meaningfully to your overall assessment plan.

On-screen: 

Bill: Norm-referenced scores – or scaled scores – show how your students compare with their peers. Scaled scores promote accurate longitudinal analyses even across different versions of the test, and help institutions make meaningful comparisons between groups of students. All Major Field Tests titles report a scaled Total Score. Many titles also report Subscores in broad subdiscipline categories. 

On-screen: Scaled scores:

Bill: Sometimes a major field can be divided into so many subdomains that a test of limited length cannot generate adequate information to report a scaled score for a subdomain. For this reason, many Major Field Tests also report a group-level score called an Assessment Indicator.

Assessment Indicators allow institutions to make internal and external comparisons in highly specific subdomains of the discipline, and help you identify areas of strength, or opportunities for improvement in your curriculum

On-screen: Assessment Indicators:

Bill: The Major Field Tests offer a broad range of reporting formats to serve your analytical needs. They feature a variety of score types and descriptive statistics about your cohort of students. All of these reports are available to you through an online assessment management portal known as the ETS Program Workshop. 

You can access results of all tests administered at your institution anytime, anywhere, whether you choose paper-and-pencil or online test administration. You can generate a report based on hundreds or even thousands of students in mere seconds. A range of options and filters allows you to tailor your report to your specific needs and generate the report in several useful formats, including Microsoft Word and Excel. 

On-screen: Standard Reports

Bill: For example, you can combine results from multiple cohorts or across years to create one consolidated report for your accreditation visiting team. Or, you can add subgroup questions to the test, and use the ETS Program Workshop to filter results on those subgroups. 

Information from Major Field Test reports can be copied into your own reports for your stakeholders. You don't need to be a data analyst or institutional researcher to begin using the scores from the Major Field Tests immediately. The standard reports are available as free downloads.

On-screen: 

Bill: In addition to the valuable standard reports, you can also subscribe to special reports that give you even more analytical power. You can run these reports as often as needed for the life of the subscription. If you test more students, simply update your reports. 

On-screen:  Special Reports

Bill: ETS Major Field Tests allow you to compare your students to those at other institutions. Comparative data is updated annually and includes all institutions that have administered each test. It is accumulated over multiple years to ensure the accuracy and representativeness of the reference group.

On-screen:

Bill: But what if you need to refine your reference group to a list of institutions and programs similar to yours? Subscribe to the Custom Comparative Data Report to generate comparison tables based only on those institutions and programs of interest to you. A peer reference group, tailored to your needs, can aid you in your accreditation efforts.

On-screen: Custom Comparative Data Report tables, based on a peer reference group, can aid you in your accreditation efforts. Selectable fields include the ability to an Institution Name and specify one or more States.

Bill: The Item Information Report provides you with item-level detail about each question on the Major Field Tests: how your students performed on each question, how the nation performed on each question, what each question was designed to measure, and what scores on the test each question contributes to. This report is the perfect tool to identify areas of a program where your students may need additional assistance.

On-screen:

[The sample report shown is a table with 7 columns. From left to right, the columns are Section, Item Number, Percent Correct Institution, Percent Correct National, Percent Omit, Percent Not Reached, Domain, Content Area, SubContent Area, Item Mapping]

Bill: What if you need to do an analysis that requires combining Major Field Test data with other data about your students? Simply subscribe to the Design Your Own Analysis service, which lets you export all of the data about your students and their performance on the Major Field Tests in easy-to-use CSV format. Import this data into your own databases to compare Major Field Test scores to other indicators, such as GPAs, admission test scores, or ePortfolio data, or open the file in Excel, where you can easily sort or tabulate on the clearly labeled data fields.

On-screen: Design Your Own Analysis service:

Bill: The ETS Major Field Tests put analytical power at your fingertips by providing you with actionable data to help you meet accreditation and program requirements.

On-screen: Analytical power

Exit [0:19]

Bill: To learn more about how the ETS Major Field Tests can help you evaluate students' mastery of their major field of study, and use that data to prove and improve  the effectiveness of your programs, please fill out our contact form. ETS welcomes your thoughts on how we can meet your assessment needs. Thank you!

On-screen: Gain the Major Field Tests Advantage [3 buttons]

Thank you for viewing the ETS Major Field Tests tour!

[END]