Development of a General Legal Confidence Scale: A First Implementation of the Rasch Measurement Model in Empirical Legal Studies

• Published date: 01 March 2019
• Date: 01 March 2019
• DOI: http://doi.org/10.1111/jels.12212

Journal of Empirical Legal Studies

Volume 16, Issue 1, 143–174, March 2019

Development of a General Legal

Confidence Scale: A First Implementation

of the Rasch Measurement Model in

Empirical Legal Studies

P. Pleasence*and N. J. Balmer

Legal capability has long been of evident importance in our understanding of legal prob-

lem resolution behavior. Although legal capability remains a contested concept, there is

much commonality betwe en specifications. Some aspects are generic, whi le others—such

as legal confidence—are pa rticular to law. Such law-specific measures a s have been devel-

oped to date have been deve loped in an ad-hoc fashio n; with no attempts made to test

psychometric propert ies, using either clas sical test theory or mo dern psychometric

methods. This has been a sh ortcoming in the empirical legal field, weake ning theoretical

development and precluding reliable estimation of changes in levels of legal capability

over time. In this articl e, we set out details of a study a imed at introducing new me thods

to scale development in t he field of empirical legal studies based on the app roaches that

have evolved in other fiel ds and the latest developme nts in psychometric modeling. Spe-

cifically, we set out deta ils of the use of a specially de signed item pool—based o n an

increasingly demanding legal sce nario—and Rasch analysis t o develop a general legal

confidence scale. Once t he 12 item pool items were re duced to a final set of six, t his

yielded a scale with good psychometric pro perties: a General Lega l Confidence (GLC)

Scale. The scale showed good ov erall fit, item fit, person fit, t argeting, and internal con -

sistency. All items had or dered thresholds, there wa s no response dependence, i tems

were unidimensional, a nd there was no evidence of diff erential item functioni ng. The

GLC scale constitutes an e ffective measure of gene ral legal confidence, an d demon-

strates it is possible to a rrive at robust and cohere nt law-specific measure s of legal capa-

bility through the careful design of questions and application of the latest psychometric

modeling techniques.

*Address correspondence to Professor Pascoe Pleasence, Faculty of Laws, University College London, Bentham

House, Endsleigh Gardens, London, WC1H 0EG, UK; email: p.pleasence@ucl.ac.uk. Balmer is at the Faculty of

Laws, University College London and Victoria Law Foundation.

We are very grateful to The Legal Education Foundation, and in particular Matthew Smerdon and Dr Natalie

Byrom, for funding the research and for their continued support for our work. We also thank the editor, Dawn

Chutkow, and anonymous referees for their carefully considered and thorough reviews, which significantly

enhanced the research.

143

I. Introduction

Galanter (1976:936) argued that “lack of capability poses the most fundamental … barrier

to access [to legality],” and Felstiner et al. (1981) that the ability to “name,” “blame,” and

“claim” is central to the emergence and transformation of legal disputes. Thus, legal capabil-

ity has long been of evident importance in our understanding of legal problem resolution

behavior. Moreover, continuing processes of “juridification,”

1

the emergence of disruptive

technologies and new forms of legal service delivery,

2

limits to legal aid,

3

policy and opera-

tional refocusing on legal service users’ (rather than providers’) needs,

4

and generally

greater understanding of factors influencing legal problem resolving behavior

5

have acted

to increase interest in both understanding legal capability and, beyond that, measuring it.

While the concept of legal capability remains contested, it links closely to Sen’s

(1999, 2002, 2010) capability approach to disadvantage and refers to the capabilities nec-

essary for a person “to resolve legal problems effectively” (Coumarelos et al. 2012:29).

Legal capability is therefore multidimensional; incorporating a range of more narrowly

framed capabilities, across a variety of domains.

Although specifications of legal capability have differed, there is much commonal-

ity. For example, Parle’s (2009), Collard et al.’s (2011), and Pleasence et al.’s (2014) spec-

ifications all incorporated knowledge of law, the ability to spot legal issues, awareness of

legal services, understanding of and the ability to assess dispute resolution options, plan-

ning and management skills, communication skills, confidence and emotional fortitude

within a series of similarly formulated domains.

6

There is also much commonality

1

Defined by Habermas (1987:357) as “the tendency towards an increase in formal (or positive, written) law that

can be observed in a modern society.”

2

See, for example, Susskind (2008) and Smith (2014).

3

Or even cuts, such as those brought about by the Legal Aid, Sentencing and Punishment of Offenders Act 2012

in England and Wales.

4

See, for example, Legal Services Commission (2006), Commonwealth Attorney-General’s Access to Justice Task-

force (2009), and Canadian National Action Committee on Access to Justice in Civil and Family Matters (2013).

5

See, for example, Pleasence and Balmer (2014) and Pleasence et al. (2015).

6

Parle’s (2009) small-scale qualitative study of young people’s legal capability suggested six core domains: knowing

rights and remedies;spotting a legal issue; knowingwhere to go for help; planning how to resolvethe issue; communi-

cating effectively; and managing emotions. Collard et al.’s (2011)later review listed 22 separate aspects of legal capa-

bility across four similarly defined domains: recognizing and framing the legal dimensions of issues and situations

(including knowledge of law and the ability to frame and explain situations in terms of the law); finding out more

about the legaldimensions of issues and situations(including the ability to identifyassess sources of legal information

and dispute resolution processes);dealing with law-related issues (including confidence and the ability to communi-

cate, plan, manage, and assess processes and outcomes); and engaging and influencing (including awareness of

means to achieve change and the confidence and strength to effect change). More recently still, Pleasence

et al. (2014) detailed a similar range ofaspects, albeit within more abstract and broadly recognized domains:knowl-

edge (of, e.g.,law, legal services, and disputeresolution options);skills (including communication skillsand the ability

to identify legal issues); attitudes (to, e.g., law itself, legal services, legal institutions, and legal systems); attributes

(such as confidence, emotionalfortitude, and preparednessto act); and resources (includingfinancial, technological,

144 Pleasence and Balmer

between specifications of legal capability and those of (more narrowly construed, but con-

ceptually similar) lawyer competence.

7

Evidently, some aspects of legal capability—such

as communication skills—are largely generic, while others—such as confidence in one’s

ability to effectively address “justiciable”

8

problems—are particular to law. In relation to

many generic aspects of legal capability, robust measures have been developed in other

fields. However, an enormous challenge remains to develop appropriate law-specific capa-

bility measures. Such measures of legal capability as exist have been developed in an ad-

hoc fashion, with no attempts made to test psychometric properties using either classical

test theory or modern psychometric methods. This is a major shortcoming. At a time

when the profile of access to justice policy has never been higher—with the adoption of

U.N. Sustainable Development Goal 16.3 (encompassing the target of access of justice for

all)

9

and growing recognition of links between access to justice and inclusive and sustain-

able growth (e.g., OECD & Open Society Foundations 2016)—the absence of robust mea-

sures of legal capability hampers the investigation of increasingly evident links between

legal capability and legal problem resolution behavior (e.g., Sandefur 2007; Gramatikov &

Porter 2011; Pleasence & Balmer 2014; Pleasence et al. 2015), and precludes effective

quantitative outcome evaluation of efforts to increase legal capability among particularly

disadvantaged populations (such as, in the United Kingdom, the Making Our Rights Reality

initiative led by Youth Access, or the work of Law for Life and Young Citizens more gener-

ally

10

). The absence of robust measures also weakens theoretical development in the area

and social). Similar, though less extensive, conceptualizations have also been offered by, for example, Balmer

et al. (2010), Coumarelos et al. (2012),and the Canadian National Action Committee on Access to Justicein Civil and

Family Matters(2013). For example, theCanadian National Action Committee on Access to Justicein Civil and Family

Matters (2013)defined legal capability as “the levelof knowledge, skills and confidenceas well as the attitudes of peo-

ple that allow them to:recognize that there are legal componentsor aspects to many activities and eventsof everyday

life; better anticipate and manage these components; be able to sort legal from non-legal aspects of their problems

and address theirinterdependence; avoid unnecessary escalationof conflicts into more serious problems or disputes

that may requirelegal intervention; assess optionsthat are available and that fosterreasonable solutions in situations

of conflict; be awareof when and how legal representationcan assist with disputes and how to accesslegal representa-

tion.” Seealso similar conceptualizations in relationto financial capability (e.g.,Kempson et al. 2005).

7

For example, Sherr et al. (1994) summarized the elements of competent lawyering as being: legal knowledge;

practical skills (e.g., interviewing, negotiating, drafting); administrative skills (e.g., practice management, supervi-

sion); motivation (to perform in an effective manner); proficiency to plan and prepare; mental and physical facul-

ties; and understanding of limitations. Similarly, Shultz and Zedeck’s (2011) extensive study identified aspects of

lawyer effectiveness to include: intellectual and cognitive (e.g., analysis, reasoning, problem solving, judgment);

research and information gathering (e.g., researching the law, fact finding, and questioning); communication

(e.g., influencing, writing, speaking, listening); planning and organizational (e.g., strategic planning, self-manage-

ment); conflict resolution (negotiation, empathy); character (e.g., diligence, stress management).

8

Justiciable problems are problems that raise legal issues, regardless of whether this is recognized by those facing

them, or whether any action taken to deal with them involves lawyers or the wider legal system (Genn 1999).

9

U.N. General Assembly Resolution 70/1, 25th September 2015. The U.N. Sustainable Development Goals build

on the earlier Millennium Development Goals and are collectively directed toward ending poverty, ensuring eco-

nomic prosperity, and sustaining the environment.

10

See, for example, http://www.lawforlife.org.uk/public-legal-education/ and https://smartlaw.org.uk/lawyers-in-

schools/ (accessed March 12, 2018).

Development of a General Legal Confidence Scale 145