When Your Plate Is Already Full: Efficient and Meaningful Outcomes Assessment for Busy Law Schools

• Citation: Vol. 71 No. 2
• Publication year: 2020

When Your Plate is Already Full: Efficient and Meaningful Outcomes Assessment for Busy Law Schools

Melissa N. Henke

[Page 529]

When Your Plate is Already Full: Efficient and Meaningful Outcomes Assessment for Busy Law Schools

by Melissa N. Henke^*

I. Introduction

The American Bar Association (ABA) accreditation standards involving outcome-based assessment are a game changer for legal education.¹ The standards reaffirm the importance of providing students with formative feedback throughout their course of study to assess and improve student learning. The standards also require law schools to evaluate their effectiveness, and to do so from the perspective of student performance within the institution's program of study. The relevant question is no longer what are law schools teaching their students, but instead, what are students learning from law schools in terms of the knowledge, skills, and values that are essential for those entering the legal profession. In other words, law schools must shift their assessment focus from one centered around inputs to one based on student outputs.

[Page 530]

Compliance with the ABA's assessment mandate comes at a time when law school resources are spread thinner than ever. Indeed, faculty already work with plates that are full with students, scholarship, and service. Thus, while not all in the legal academy are on board with the ABA's approach to outcomes assessment or to outcomes assessment generally, as busy educators, we should all at least agree that the requisite response should be efficient, given that resources are limited, and meaningful, such that the work done can benefit our learners.² To do so, law schools should begin at their own tables set with full plates, so to speak, taking stock of what institutions and their faculty are already doing in terms of assessment. And it is important to think broadly here, as faculty may be surprised to learn how many of their colleagues are already doing relevant work.

While law schools may already be inclined to begin from within, this Article outlines concrete strategies they can use when working with existing faculty expertise and resources to respond to the ABA's assessment mandate in a meaningful way for students, and with the goal of maximizing efficiency and gaining broad buy in. While prior scholarship has outlined best practices for outcomes assessment and even shared examples of how to engage in the process in the law school setting, this Article is unique in its depth and breadth of coverage by setting out a detailed case study³ that illustrates the process of developing an authentic assessment tool and beginning the process for adapting that tool to respond to both the individual student assessment and law school assessment required by the ABA.

To be clear, this Article does not suggest that only those with existing expertise or resources should be the ones to actually engage in the outcomes assessment work now required by the ABA. The goal should not be to add to the plates of a few. Instead, to create a productive and meaningful culture of assessment, experts in the field proclaim that administrators and faculty must all be involved.⁴ The ABA agrees.⁵ In

[Page 531]

addition to encouraging broad buy in, a more collaborative approach helps ensure that assessment work is equitably spread among faculty.

Part II reviews the ABA standards relevant to outcomes assessment, discussing the two types of outcomes assessment required by those standards—individual student assessment and law school assessment—and sharing the underlying theory behind both. Part III outlines the stages of outcomes assessment, with a specific focus on the measurement stage of the process, because it is arguably the most time-intensive stage of the process and the one in which existing resources can prove most valuable. Part IV focuses on one common direct assessment measure, the analytic rubric, detailing how UK Law's legal writing faculty collaboratively designed a rubric for the LRW Course appellate brief assignment, and responding to concerns that have been raised about using rubrics for assessment. Finally, Part V provides specific suggestions on how to adapt and use existing assessment measures most efficiently when responding to the ABA's assessment mandate at both the individual student and law school levels. In other words, assessment measures, like the rubric project described in Part IV, can be adapted and used more broadly than the purpose for which they were originally designed. While the LRW Course appellate brief assignment rubric serves as the primary example to illustrate these ideas, this Article will touch on other examples and share ideas about how a variety of existing resources can transfer to the current assessment landscape mandated by the ABA.

The message here is that law schools need not panic, as they are likely to find they have more relevant assessment knowledge and

[Page 532]

materials to work from than first thought. If professors are willing to share their relevant experience and resources, work collaboratively to expand and adapt from that base as needed, and spread the related assessment responsibilities widely and fairly among the faculty, then the ABA's call for outcomes assessment can be answered with meaning and without forcing any one faculty member's plate to overflow.

II. The ABA Standards on Learning Outcomes, Formative Assessment, and Institutional Assessment

This Part offers general background on the ABA standards relating to learning outcomes and assessment. Section B then follows with a more in-depth look at the theory behind the types of assessment law schools must engage in under the described standards.

A. The Relevant ABA Standards

In 2008, the Council of the Section of Legal Education and Admissions to the Bar charged the Standards Review Committee to lead a comprehensive review of the accreditation standards governing legal education. Two important components of the review are the Special Committee on Output Measures and the Student Learning Outcomes Subcommittee (Output Measures Committee). The Output Measures Committee was charged with determining "whether and how output measures, other than bar passage and job placement, might be used in the accreditation process."⁶ The focus historically had been on a law school's inputs, in terms of resources invested into the educational process, and on indirect output data regarding bar passage and job placement rates.⁷ The Output Measures Committee issued a seventy-one-page report analyzing how other accreditation bodies use outcomes measures (all ten of the other professional accrediting bodies reviewed used outcome measures in their standards) and noting that regional accreditation agencies have also been focused on student learning

[Page 533]

outcomes.⁸ The report concluded that current ABA accreditation standards should be reviewed and revised "to reduce their reliance on input measures and instead adopt a greater and more overt reliance on outcome measures."⁹ The Standards Review Committee responded by studying the matter and making recommendations to the Council, which included input from the Student Learning Outcomes Subcommittee.

The Standards Review Committee recommendations resulted in new and revised standards adopted by the Council, which went into effect on August 12, 2014. The most relevant standards for this Article are Standards 301, 302, 314, and 315.¹⁰

As they relate to this Article, the Assessment Standards set out new requirements regarding learning outcomes and assessment. A key

[Page 534]

guiding principle in the implementation of the standards is that "[t]he focus on outcomes should shift the emphasis from what is being taught to what is being learned by the students."¹¹ Generally speaking, the goal of "outcomes assessment is to understand how educational programs are working and to determine whether they are contributing to student growth and development."¹² An example I used with my faculty colleagues considers when a parent tells her child to feed the dog each morning before leaving for school. Inputs assessment measures effectiveness simply by looking to what the parent said to the child about feeding the dog (morning reminders, a written note on the refrigerator). However, outcomes assessment shifts the focus to the results of those reminders by looking to whether there is actually food in the dog's bowl each morning. It is not enough to just claim success by "teaching" the child to feed the dog if the results show that the child has not actually learned to complete the task and the dog is left hungry.

While outcomes assessment is new for law schools, it is unlikely to be a fleeting trend in legal education.¹³ Many view the change as a positive and long overdue one for legal education, and one that law schools can truly benefit from.¹⁴ According to proponents, outcomes assessment promotes active student learning, which can better prepare students to enter the legal profession, and to do so as more self-directed learners.¹⁵ They say it also promotes reflective teaching, which can result in important curricular changes where needed.¹⁶ But not everyone in the academy has been so quick to embrace the Assessment Standards and

[Page 535]

related changes, especially given the time and resources involved.¹⁷ Regardless of one's view on their merit, the Assessment Standards have been described as "the most significant change in law school accreditation standards in decades."¹⁸ As one scholar put it, "[t]he new ABA accreditation standards reflect a 'fundamental shift' in the delivery of legal education and curricular design . . . ."¹⁹ Others have used words like "revolutionary" and "sea change."²⁰

There are two key components to the ABA's assessment mandate. First, law schools must engage in formative assessment in addition to summative assessment, at least in some courses, to inform individual student learning. Second, each accredited...