Human Capital and International Large-Scale Assessment

As societies become increasingly advanced, adults are expected to use information in more complex ways and to maintain and enhance their skills throughout their lifetimes. Such skills are critical not only for personal achievement, but also for positive social, educational and economic outcomes around the world.

ETS's research in this area is based on international large-scale comparative surveys of both adults and school-age youth. The focus is on core skills such as reading, math/numeracy and science literacy. Other areas addressed in these surveys include innovative constructs such as financial literacy, collaborative problem solving and problem solving in technology-rich environments.

Projects

ETS's research in human capital and international large-scale assessment is conducted both in the United States and around the world. Our large-scale international comparative surveys are designed to meet policy needs reflecting a growing interest in skills as human capital. These are some examples:

OECD Survey of Adult Skills
ETS manages this major assessment for the Organisation for Economic Co-operation and Development (OECD) as part of the Programme for the International Assessment of Adult Competencies (PIAAC). This is the first large-scale, computer-delivered comparative assessment of adults in the world. It provides policymakers with a profile of adults' knowledge, skills and competencies in literacy, numeracy, and problem solving in technology-rich environments. PIAAC expands upon earlier literacy assessments, and is the first multistage adaptive test used in large-scale comparative assessment.

We also work on two other projects related to the PIAAC program:

  • Education and Skills Online — A new computer-delivered assessment linked to PIAAC. It provides individual results for persons who want to demonstrate readiness skills, and allows institutions, organizations and local governments to assess a population for training and research purposes.
  • The World Bank's Skills Toward Employment and Productivity (STEP) — This program, which is linked to PIAAC, measures reading literacy and component skills, which include vocabulary knowledge, sentence processing and passive comprehension. It seeks to help countries assess how different skill sets affect labor-market opportunities, and provides information on employability and marketability.

In November 2013, ETS organized the PIAAC Invitational Research Conference, which highlighted the importance of the PIAAC results for understanding the role of skills in a global context. Watch the conference's opening video.

Programme for International Student Assessment (PISA)
ETS manages and develops this global evaluation of education systems for the OECD. PISA is a multinational survey of 15-year-old students, currently administered in some 70 countries. The test, which transitioned from paper to computer in 2015, assesses what students know and their ability to apply their knowledge in real-life situations at an age when they begin to fully participate in society.

TIMSS and PIRLS
ETS acts as consultant to Boston College on two international assessments: Trends in International Mathematics and Science Study, as well as the Progress in International Reading Literacy Study.

IEA-ETS Research Institute (IERI)
ETS collaborates with the International Association for the Evaluation of Educational Achievement (IEA) on research studies, dissemination efforts, and professional development focused on improving the science of large-scale assessments. IERI publishes an international journal and runs workshops on methodology using existing large-scale assessment databases. Both independently and as part of its involvement in IERI, ETS's researchers also produce research reports, books and book chapters related to human capital and international large-scale assessment (see list below).

International Large-Scale Assessment Conference
Designed to present and discuss multidisciplinary issues related to the use and implementation of international large-scale assessments, the 2011 event informed funders, policymakers, managers and technical staff of existing international large-scale assessment programs.

Read the publication that resulted from the conference: The Role of International Large-Scale Assessments: Perspectives from Technology, Economy, and Educational Research.

Featured Publications

Here is a selection of significant publications related to human capital and international large-scale assessment:

2016

  • Choosing Our Future: A Story of Opportunity in America
    I. S. Kirsch, H. Braun, M. L. Lennon, & A. Sands
    An ETS report

    Over the past 40 years, technological changes, interactions among powerful global economic forces, government policies and business practices have generated a self-sustaining set of dynamics that have altered the workplace, the demands of everyday life, and our civic and social lives. The authors analyze the complex and self-sustaining dynamics governing the distribution and intergenerational transmission of opportunity with the goal of advancing our national conversation about the increasing polarization of opportunity and why we must take action.

  • The Dynamics of Opportunity in America: Evidence and Perspectives
    Editors: I. Kirsch & H. Braun
    Publisher: Springer

    Our children are beginning life from very different starting points that place them on distinctly different trajectories of growth and development, leading to vastly different adult outcomes. Leaders from fields including education, economics, demography and political science illuminate these dynamics that are converging to create a cycle of opportunity inequality in the United States, and offer suggestions for reversing these harmful trends.

2015

2014

2013

2012

2011

  • An Experimental Study of the Effects of Monetary Incentives on Performance on the 12th-Grade NAEP Reading Assessment
    H. Braun, I. Kirsch, & K. Yamamoto
    Teachers College Record, Vol. 113, No. 11, pp. 2309–2344

    This article reports on a study of monetary incentives for 12th-grade students taking an assessment designed to replicate the NAEP reading test. The study concluded that NAEP may both underestimate the reading abilities of students enrolled in 12th grade and yield biased estimates of certain achievement gaps. View publisher's page >

  • Measuring Growth in a Longitudinal Large Scale Assessment with a General Latent Variable Model
    M. von Davier, X. Xu, & C. H. Carstensen
    Psychometrika, Vol. 76, No. 2, pp. 318–336

    This article presents approaches for use with longitudinal large-scale assessment data, i.e., data that were repeatedly collected on the same sample of students or adults. The article describes models that made it possible to examine whether the Matthew-effect (those who have more, gain more) can be identified in a national extension of the PISA assessment. View citation record >

  • Investigation of Model Fit and Score Scale Comparability in International Assessments
    M. E. Oliveri & M. von Davier
    Psychological Test and Assessment Modeling, Vol. 53, No. 3, pp. 315–333

    This article presents evidence on scale stability and the need to account for national variations in how some tasks in international assessments are understood. While rigorous translation, verification and assessments of cultural appropriateness reduce the threat of cross-country differences in how tasks are understood, it is always possible to detect deviations from the international standard on a limited number of tasks. This article presents an approach of how such deviation can be remedied. View article >

2010

  • Relationships Among Reading Skills of Adults with Low Literacy
    J. P. Sabatini, Y. Sawaki, J. R. Shore, & H. S. Scarborough
    Journal of Learning Disabilities, Vol. 43, No. 2, pp. 122–138

    In this study, researchers used a method known as confirmatory factor analysis to examine the relationship between literacy and several component reading skills (e.g., word recognition, fluency, language comprehension and vocabulary skills) in adults. View publisher's page >

2009

2008

  • Preparing for an Epidemic of Limited Health Literacy: Weathering the Perfect Storm
    R. M. Parker, M. S. Wolf, & I. Kirsch
    Journal of General Internal Medicine, Vol. 23, No. 8, pp. 1273–1276

    Research has demonstrated strong links between literacy and health. ETS's recent report America's Perfect Storm considers the great risks the United States faces as a result of declining adult literacy, shifting demographics and a changing economy. It is essential to understand how these changes will impact health care. View publisher's page >

2007

  • America's Perfect Storm: Three Forces Changing Our Nation's Future
    I. Kirsch, H. Braun, K. Yamamoto, & A. Sum
    ETS Policy Information Report

    This report looks at the convergence of three powerful sociological and economical forces that are changing the future of the United States: substantial disparities in skill levels (reading and math), seismic economic changes (widening wage gaps), sweeping demographic shifts (less education, lower skills). View summary >

  • Adult Learners in a Changing America
    I. Kirsch, M. Lennon, & C. Tamassia
    in B. Guzzetti (Editor), Literacy for the New Millennium, Vol. 4, pp. 91–122 Publisher: Praeger

    This ETS-authored article appears in a multivolume series on literacy research. View citation record >

2005

  • Learning a Living: First Results of the Adult Literacy and Life Skills Survey
    Organisation for Economic Co-operation and Development & Statistics Canada

    This report on the Adult Literacy and Life Skills survey (ALL) presents evidence on the nature and magnitude of the literacy gaps faced by OECD countries and how these gaps evolved over the previous decade. It contains interpretation of data for the seven countries participating in the first round of data collection. View the report here >

2004

1997–2002