Multiple-Choice: Choosing the Best Options for More Effective and Less Frustrating Law School Testing

AuthorJanet W. Fisher
PositionAssociate Professor of Academic Support, Suffolk University Law School
Pages119-136

Page 119

MULTIPLE-CHOICE: CHOOSING THE BEST OPTIONS FOR MORE EFFECTIVE AND LESS FRUSTRATING LAW SCHOOL TESTING

JANET W. FISHER*

I. INTRODUCTION

“I always have trouble with the multiple-choice questions,” is a common law student complaint to academic support faculty. Many of the students who seek our assistance are distraught over their failure to perform well on multiple-choice questions because performing poorly on even one test can have serious consequences in the high-stakes testing environment of law school. Doctrinal faculty occasionally express puzzlement to academic support faculty over why some students perform poorly on multiple-choice questions and may refer these students to academic support faculty for help. Academic support faculty find it difficult to give helpful advice to students because these professionals usually are working in the dark as to how doctrinal faculty construct multiple-choice questions. The multiple-choice testing experience could be improved for both faculty and students through adherence to multiple-choice item-writing guidelines and use of appeal or answer justification procedures. These practices would yield more effective and less frustrating testing experiences for both faculty and students.

This article discusses various means by which multiple-choice testing in law school could be improved. Part II provides background on multiple-choice testing. Part III describes problems that arise for students and faculty in connection with multiple-choice testing. Part IV examines possible effects of flawed multiple-choice questions. Part V reviews basic multiple-choice item-writing guidelines. Part VI reviews some general principles of test validity. Finally, Part VII evaluates appeal and answer

_______________________________________________________

Copyright © 2008, Janet W. Fisher.

* Associate Professor of Academic Support, Suffolk University Law School; B.A. Fontbonne College; J.D. Suffolk University Law School. The writer thanks Donna Qualters, Director of the Suffolk University Center for Teaching Excellence, and Professor Herbert Ramy, Professor Kathleen Elliott Vinson, and Legal Reference Librarian Diane D’Angelo of Suffolk University Law School for their contributions to this article.

Page 120

120 CAPITAL UNIVERSITY LAW REVIEW [37:119

justification procedures that could be used to supplement multiple-choice testing.

II. BACKGROUND

Multiple-choice testing, which is widely accepted today as part of the academic landscape, is less than 100 years old.1The format was developed as an efficient way to administer intelligence tests to the U.S. Army during World War I.2Stanford graduate student Arthur S. Otis is recognized for his contribution to development of the Army intelligence tests.3Otis’s work, in turn, was influenced by the Kansas Silent Reading Test, developed in 1914 through 1915 by Frederick J. Kelly, which is believed to contain the first known publication of multiple-choice questions.4

Use of the new format spread quickly. By 1926 the College Entrance Examination Board had added the multiple-choice SAT (Scholastic Aptitude Tests) to its test offerings.5The LSAT (Law School Admission Test) was administered for the first time in l948.6The 1948 test took a full day and consisted of ten sections.7Most of the sections were in multiple-choice format.8While many changes have been made to the LSAT over

_______________________________________________________

1Franz Samelson, Was Early Mental Testing: (a) Racist Inspired, (b) Objective Science, (c) A Technology for Democracy, (d) The Origin of the Multiple-Choice Exams, (e) None of the Above? (Mark the RIGHT Answer), in PSYCHOLOGICAL TESTING AND AMERICAN

SOCIETY 1890–1930 113, 115–16 (Michael M. Sokal ed., 1987).

2Id. at 116.

3Id. at 116–17.

4Id. at 118–19.

5Id. at 122. The SAT as currently administered includes not only multiple-choice questions but also a student-produced essay and other questions that call for student-produced responses. CollegeBoard, SAT Reasoning Test, http://www.collegeboard.com/ student/testing/sat/about/SATI.html (last visited Dec. 9, 2008).

6LYNDA M. REESE & RUTH ANNE COTTER, A COMPENDIUM OF LSAT AND LSACSPONSORED ITEM TYPES, 1948–1994 1 (1994), http://www.lsacnet.org/Research/

Compendium-of-LSAT-and-LSAC-Sponsored-Item-Types.pdf.

7Id. (listing those sections as Verbal Analogies, Sentence Completion, Paragraphs, Word Classification, two sections of Reading Comprehension, Figure Classification, Debates, Contrary and Irrelevant Statements and Reasoning).

8Id. at 29 app.

Page 121

2008] MULTIPLE-CHOICE 121

the years, the test is still primarily in multiple-choice format.9The

multiple-choice format also influenced the bar examination process. The National Conference of Bar Examiners created the Multistate Bar Exam (MBE) in the early l970s.10The MBE has been a multiple-choice test since its inception.11

At the time the MBE was created, law school exams had been dominated by hypothetical essay questions for decades.12In response to developments in testing theory, true/false questions were used in law school exams as early as the 1920s, and there are examples of multiple-choice questions on law school exams from the 1950s.13With the

inception of the MBE, interest in multiple-choice testing grew,14and some law faculty began to believe that multiple-choice testing in law school would help prepare students for the MBE.15It is likely that law faculty

_______________________________________________________

9The LSAT as currently administered is a half-day test consisting of five sections of multiple-choice questions, plus a writing sample which is administered at the end of the test. The writing sample is not scored but is sent to each law school to which the examinee has applied. Law School Admission Council, About the LSAT, http://www.lsac.org/LSAT/ about-the-lsat.asp (last visited Dec. 15, 2008).

10John Eckler, The Multistate Bar Examination: Its Origins and Objectives, B. EXAMINER, Feb. 1996, at 14, 16–17.

11Id. at 17 (noting that the original MBE consisted of 200 multiple-choice questions, covering five subject areas). The MBE as currently administered consists of 200 multiple-choice questions covering six subject areas. National Conference of Bar Examiners, The MultiState Bar Examination (MBE), http://www.ncbex.org/multistate-tests/mbe/ (last visited Dec. 15, 2008).

12See Steve Sheppard, An Informal History of How Law Schools Evaluate Students, with a Predictable Emphasis on Law School Final Exams, 65 UMKC L. REV. 657, 677–79 (1997).

13Id. at 682–84.

14Id. at 684.

15See 1 MICHAEL S. JOSEPHSON, LEARNING AND EVALUATION IN LAW SCHOOL: PRINCIPLES OF TESTING AND GRADING, LEARNING THEORY AND INSTRUCTIONAL OBJECTIVES

39 (1984) [hereinafter VOL. 1 JOSEPHSON] (“Since the vast majority of law school graduates will be required to take the MBE, student’s (sic) who have been given no experience with this objective format in their three or four years of law school will be at a distinct disadvantage on the bar exam.”); Howard J. Gensler, Valid Objective Test Construction, 60 ST. JOHN’S L. REV. 288, 288–89 (1986) (“The objective law school test is valuable both for evaluating the law student and exposing him to multiple[-]choice questions prior to the bar examination.”) (citation omitted).

Page 122

122 CAPITAL UNIVERSITY LAW REVIEW [37:119

also were attracted to multiple-choice testing because it allowed them to include a greater variety of issues in any one exam with the added bonus of ease of grading.16

III. THE PROBLEMS

A variety of problems can arise for students and faculty in connection with multiple-choice testing. Students may incorrectly assess what is required to perform well on multiple-choice questions. Academic support faculty may lack adequate information to advise students how to approach multiple-choice questions. Students who perform poorly on multiple-choice questions are often frustrated and anxious and may want to meet with doctrinal faculty who, in turn, may be puzzled about why particular students performed poorly on the questions.

When students arrive at law school, their most recent experience with multiple-choice testing usually has been the LSAT. Having been successful enough on the LSAT to be admitted to law school, many students believe that the same preparation and test-taking strategies that worked on the LSAT will serve them well on law school multiple-choice exams. Many students describe a multiple-choice test-taking strategy that consists of eliminating two options that “must be wrong” and then reasoning their way to the correct choice between the remaining two options.

Many students do not fully take into account that the LSAT is fundamentally different from law school multiple-choice exams. The LSAT tests general verbal reasoning skills and does not require mastery of any particular body of knowledge.17Law school multiple-choice questions, by contrast, are intended to test mastery of legal rules and

_______________________________________________________

16 Gensler, supra note 15, at 289, 294.

17According to the Law School Admission Council:
The LSAT is designed to measure skills that are considered essential for success in law school: the reading and comprehension of complex texts with accuracy and insight; the organization and management of information and the ability to draw reasonable inferences from it; the ability to think critically; and the analysis and evaluation of the reasoning and arguments of others.

LSAC.org, About the LSAT, http://www.lsat.org/LSAT/about-the-lsat.asp (last visited Dec. 9, 2008).

Page 123

2008] MULTIPLE-CHOICE 123

concepts. While students seem to understand this difference when it is explained, many have incorrectly assumed that a general familiarity with the legal rules, coupled with their...

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT