Advances in Measurement and Cognition

• Date: 01 May 2019
• DOI: 10.1177/0002716219843816
• Published date: 01 May 2019
• Subject Matter: Toward the Future: Theories of Knowing and Implications for Assessment

164 ANNALS, AAPSS, 683, May 2019

DOI: 10.1177/0002716219843816

Advances in

Measurement

and Cognition

By

ROBERT J. MISLEVY

843816ANN The Annals of The American AcademyAdvances in Measurement and Cognition

research-article2019

Situative, sociocognitive (SC) psychology is forcing a

reconception of educational assessment. The SC

perspective emphasizes the interplay between across-

person linguistic, cultural, and substantive patterns that

human activity is organized around and within-person

cognitive resources that individuals develop to partici-

pate in activities. Rather than seeing assessment

primarily as measurement, we are increasingly seeing it

as an evidentiary argument, situated in social contexts,

shaped by purposes, and centered on students’ devel-

oping capabilities for valued activities. Developments

in technology and analytic methods support new prac-

tices and familiar practices as reconceived. Implications

follow for current challenges such as assessing higher-

order skills, performance in digital environments, and

diverse student populations.

Keywords: argumentation; assessment; measure-

ment; sociocognition

Educational assessments gather information

about students’ capabilities for many pur-

poses in many contexts. Various assessment

“use cases” characterize individual students’

proficiencies, guide learning in classrooms and

intelligent tutoring systems, evaluate instruc-

tional programs, and compare educational

Robert J. Mislevy is Frederic M. Lord Chair in measure-

ment and statistics at ETS and professor emeritus of

measurement and statistics at the University of Maryland,

with affiliation in second language acquisition. His

research applies developments in technology, statistics,

and cognitive science to educational assessment.

NOTE: I am grateful to the National Academy of

Education and the American Academy of Political and

Social Science for the opportunity to contribute to this

special issue; its editors, Amy Berman, Jim Pellegrino,

and Michael Feuer, for their support and feedback

throughout the process; the ETS editor, Rebecca

Zwick, and reviewers Isaac Bejar and Randy Bennett;

and my fellow authors and commenters, especially

Scott Marion, for the conversations concerning this

article and the others as a collection.

Correspondence: rmislevy@ets.org

ADVANCES IN MEASUREMENT AND COGNITION 165

systems (see Table 1). Implications for all these uses follow from advances in

cognitive and measurement sciences—specifically, a sociocognitive (SC) per-

spective on human learning and action and an argumentation framework for

designing, interpreting, and using assessments. I describe the changes and the

ways in which they are beginning to transform assessment.1

TABLE 1

Representative Assessment Use Cases

Use Case Key Features with Inferential Implications

Large-scale accountability

tests

Are not directly connected to what students are studying (“drop in from

the sky,” or DIFTS, from students’ perspective).

Address standards targeted to grades.

Samples of same tasks in the same way for all students at the same time.

Little known about student background and previous learning.

Administered at a time chosen by the user (e.g., state), same for every-

one.

Comparable across students and schools, for what can be assessed.

Limited usefulness for individual students’ learning.

Large-scale educational

surveys (e.g., the National

Assessment of Educational

Progress)

Are not directly connected to what students are studying (DIFTS).

Address content framework overlaps but need not be the same as

domain area or state/national standards.

Samples of same tasks in the same way for all students at the same time.

Administered at a time chosen by the program—same for everyone.

Little known about student background and previous learning.

Comparable across states and reporting groups, for what can be assessed.

Not useful for individual students’ learning.

Summative assessments to

assign grades/course credit

(i.e., end-of-course tests)

Are directly connected to what students have studied.

Instruction and assessment can be matched to student characteristics

(e.g., background knowledge, culture, language).

Useful to evaluate what individual students have been studying.

Results not necessarily comparable across students and schools in that

they are constructed to course goals, representations, instructional

methods, etc. The more tailoring (for improved learning!), the less

comparability.

Useful for gauging assessable-by-test learning targets, but not for finer-

grained feedback for learning along the way.

Assessments in intelligent

tutoring systems (ITS)

Are directly connected to what students are studying.

Previous experience and current options known, via student history in ITS.

Administered when and to whom it is most useful to guide learning.

Comparability with respect to students at the same location in the ITS at

a given time, but not generally at other locations or other curricula.

Formative assessments in

classroom (e.g., quizzes,

feedback during project)

Are directly connected to what students are studying.

Instruction and assessment can be tailored to student characteristics, in

particular previous learning experiences, to optimize current learning.

Administered when, how, and to whom it is most useful to guide learning.

Generally don’t support comparable inferences, even with same tasks,

because student background and histories are not known. The more

tasks and instruction are so tailored (to improved learning!), the less

comparable.