Lawyer and Judicial Competency in the Era of Artificial Intelligence: Ethical Requirements For Documenting Datasets and Machine Learning Models

Lawyer and Judicial Competency in the Era of

Artif‌icial Intelligence: Ethical Requirements for

Documenting Datasets and Machine Learning

Models

MARK L. SHOPE*

ABSTRACT

Judges and lawyers have the duty of technology competence, which includes

competence in artif‌icial intelligence technologies (“AI”). So not only must law-

yers advise their clients on new legal, regulatory, ethical, and human rights

challenges associated with AI, they increasingly need to evaluate the ethical

implications of including AI technology tools in their own legal practice.

Similarly, judge competence consists of, among other things, knowledge and

skill of technology relevant to service as a judicial off‌icer, which includes AI.

After describing how AI implicates ethical issues for lawyers and judges and

the requirement for lawyers and judges to have technical competency in the AI

tools they use, this article argues for the requirement to use one or both of the

following human interpretable AI disclosure forms when lawyers and judges

are using AI tools: Dataset Disclosure Form or Model Disclosure Form.

TABLE OF CONTENTS

INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

I. LAWYER AND JUDGE COMPETENCE . . . . . . . . . . . . . . . . . . . . 193

II. PARTICULAR COMPETENCE FOR LAWYERS . . . . . . . . . . . . . . 194

A. RELEVANT RULES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

III. PARTICULAR COMPETENCE FOR JUDGES . . . . . . . . . . . . . . . . 201

* Assistant Professor of Law, National Yang Ming Chiao Tung University School of Law (markshope@-

nycu.edu.tw). This article is based on the author’s presentation at The 2nd International Academic Conference

on Artif‌icial Intelligence and Law hosted by the International Artif‌icial Intelligence and Law Research

Foundation on December 26-27, 2019, in Taipei and Kaohsiung, Taiwan. The main topic of the conference was

“The Collision of the Legal Profession and Technology in the Modern Era: International Inquiry into AI and

Law.” © 2021, Mark L. Shope.

191

A. RELEVANT RULES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

IV. MACHINE LEARNING LIFECYCLE. . . . . . . . . . . . . . . . . . . . . . 204

V. PROBLEMS WITH ARTIFICIAL INTELLIGENCE . . . . . . . . . . . . . 206

VI. DATASET DISCLOSURE FORMS AND MODEL DISCLOSURE

FORMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

VII. DATASET DISCLOSURE FORMS. . . . . . . . . . . . . . . . . . . . . . . . 212

VIII. MODEL DISCLOSURE FORMS . . . . . . . . . . . . . . . . . . . . . . . . . 218

CONCLUSION. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

INTRODUCTION

Judges and lawyers have the duty of technology competence, which includes

competence in artif‌icial intelligence (“AI”).

1

In practice, not only must lawyers

advise their clients on new legal, regulatory, ethical, and human rights challenges

associated with AI, but they increasingly must also evaluate the ethical implica-

tions of including AI technology tools in their own legal practice.

2

Similarly,

judge competence consists of, among other things, knowledge and skill of tech-

nology relevant to service as a judicial off‌icer, which includes AI.

3

After describ-

ing how AI implicates ethical issues for lawyers and judges, and the requirement

for lawyers and judges to have technical competency in the AI tools they use, this

article argues for the use of dataset disclosure forms for datasets (“Dataset

Disclosure Forms”) and model disclosure forms for models (“Model Disclosure

Forms”) when lawyers and judges are using AI tools in their professional

capacity. Currently, there is no standardized process for documenting datasets

and models in AI.

4

If lawyers and judges are using AI tools, relevant state

1. See Agnieszka McPeak, Disruptive Technology and the Ethical Lawyer, 50 U. TOL. L. REV. 457, 457–58

(2019) (noting that the “duty of technological competence has come about at a time when innovation, often

fueled by artif‌icial intelligence, has produced new legal technology, . . . [which] require lawyers to adapt the

very tools of their trade in order to stay competent”).

2. See Catherine Nunez, Artif‌icial Intelligence and Legal Ethics: Whether AI Lawyers Can Make Ethical

Decisions, 20 TUL. J. TECH. & INTELL. PROP. 189, 195 (2017) (discussing various theories of a lawyer’s role in

legal ethics).

3. See Michael Thomas Murphy, Just and Speedy: On Civil Discovery Sanctions for Luddite Lawyers, 25

GEO. MASON L. REV. 36, 52 (2017) (observing the increases in technological competency of judges).

4. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal

Daume

´ III & Kate Crawford, Datasheets for Datasets 2 (Working Paper, No. 1803.09010v7, 2020), https://

arxiv.org/pdf/1803.09010.pdf [https://perma.cc/M4EW-ARXD]. The author would like to extend a special

192 THE GEORGETOWN JOURNAL OF LEGAL ETHICS [Vol. 34:191

professional conduct rules for lawyers and judges create the obligation to under-

stand these tools.

5

However, if AI tool providers do not divulge the inner working

of those AI tools, then how can lawyers and judges truly understand these tools?

To begin to answer this critical inquiry, this article argues for the use of Dataset

Disclosure Forms and Model Disclosure Forms to fulf‌ill lawyers’ and judges’ eth-

ical requirement to understand the tools they are using. A Dataset Disclosure

Form or a Model Disclosure Form may help stakeholders make informed deci-

sions about the use of a particular AI tool and its f‌laws; thus, such forms should

be included in relevant court procedures.

I. LAWYER AND JUDGE COMPETENCE

The ABA Model Rules of Professional Conduct (the “Model Rules”) include

the duty of technology competence in Rule 1.1, Comment 8.

6

Since AI is one

kind of technology, this technology competence would include competence in AI

technologies relevant to the practitioner’s legal practice.

7

Lawyers must evaluate

how and to what extent they are including AI technology tools in their own legal

practice.

8

For example, Model Rules 1.4 (Communications) or 1.5 (Fees) could

be triggered if a lawyer thinks using a particular AI solution would be useful

for a client’s legal needs and when the lawyer is discussing fees. Rule 1.6

(Conf‌identiality of Information) is triggered when a lawyer is using an AI solu-

tion that is not in-house, possibly exposing conf‌idential information to a third-

party (for example, when electronic document review is conducted off-site by a

third-party). Communicating with the court and opposing counsel regarding elec-

tronic discovery could implicate Rules 3.3 (Candor to the Tribunal) and 3.4

(Fairness to Opposing Party & Counsel). Rule 5.1 (Responsibilities of a Partner

or Supervisory Lawyer) and Rule 5.3 (Responsibilities Regarding Nonlawyer

Assistants) could be triggered when a subordinate is tasked with deciding which

particular AI tool to use and further while implementing those tools. Rule 5.5

thanks to the authors of this article, including Timnit Gebru, who approved the use of a portion of their ground-

breaking paper to develop the Dataset Disclosure Forms and Model Disclosure Forms found below.

5. See Marla N. Greenstein, AI and a Judge’s Ethical Obligations, 59 Judges’ J. (forthcoming Winter 2020)

(noting that for judges to comply “with their ethical responsibilities while using AI or interpreting its proper

use, [they] must f‌irst ensure that they understand the AI application involved”). This article focuses on a practi-

cal implementation of relevant model rules by discussing those rules as they are manifest in Indiana.

6. See MODEL RULES OF PROF’L CONDUCT R. 1.1 cmt. 8 (2020). [hereinafter MODEL RULES].

7. Ed Walters, The Model Rules of Autonomous Conduct: Ethical Responsibilities of Lawyers and Artif‌icial

Intelligence, 35 GA. ST. U. L. REV. 1073, 1076 (2019) (noting that “as the quality of work product created by

lawyers augmented with AI surpasses the work created without AI, it is clear that lawyers will soon have a pro-

fessional responsibility to employ new techniques”).

8. See Katherine Medianik, Artif‌icially Intelligent Lawyers: Updating the Model Rules of Professional

Conduct in Accordance with the New Technological Era, 39 CARDOZO L. REV. 1497, 1501–02 (2018) (propos-

ing, given the growing technology in the legal f‌ield, “(1) the addition of continuing legal education (CLE)

requirements on ‘Legal Technology’; (2) the addition of the term ‘nonlawyer assistant’ to the terminology sec-

tion of the Model Rules [of Professional Conduct]; and (3) the addition of several comments that incorporate

AI technology and account for technological advancement”).

2021] LAWYER AND JUDICIAL COMPETENCY IN THE ERA OF A.I. 193