What counts as evidence for policy? An analysis of policy actors' perceptions

• Published date: 01 September 2023
• Author: Eleanor MacKillop,James Downe
• Date: 01 September 2023
• DOI: http://doi.org/10.1111/puar.13567

RESEARCH ARTICLE

What counts as evidence for policy? An analysis of policy

actors’perceptions

Eleanor MacKillop | James Downe

Wales Centre for Public Policy, Spark, Cardiff

University, Cardiff, UK

Correspondence

Eleanor MacKillop, Wales Centre for Public

Policy, Spark, Cardiff University, Maindy Road,

Cardiff CF24 4HQ, UK.

Email: eleanor.mackillop@wcpp.org.uk

Abstract

Evidence plays a growing role in public administration worldwide. We analyze the

perceptions of policy actors, using Q methodology and a structured questionnaire,

which reveals four types of profiles. Most policy actors did not fit neatly into an

Evidence-Based Policy-Making (EBPM) group. Instead, they either had a pragmatic

view where context and policy issues influence what counts as evidence, an inclu-

sive position which emphasized the importance of considering a range of different

types of evidence, or a political perspective where power relations and politics

influence what counts as evidence. Our research also illustrates how different

actors in the same community can have different perceptions of evidence, and

how this can change over time due to experience and career trajectory.

Evidence for Practice

•Academic debates reflect the variety of meanings over what counts as evidence

for policy. Our article uses Q methodology to demonstrate how many policy

actors espouse a pragmatist and permissive approach to evidence.

•What counts as evidence depends on the context, the nature of the questions

that are being addressed, and the evidence that is available.

•Policy actors’perceptions of what counts as evidence often change as their

careers progress, and are influenced by the roles they take on, the organizations

they work in, and the policy issues they work on.

•Understanding the breadth and mutability of policy actors’perceptions of evi-

dence is important for both policy-makers and scholars as this can help them

better communicate and negotiate with each other.

Despite some populist pushbacks, it is often taken for

granted across many policies and political systems that

evidence ought to inform policy (Cairney et al., 2016; Van-

Landingham & Silloway, 2016). What is meant by evi-

dence, however, varies according to who is asked, the

context, and the policy problem being addressed,

amongst other factors (Crowley & Taylor Scott, 2017).

Therefore, understanding what different policy actors—

from policy-makers to producers of knowledge (e.g., think

tanks and advocacy groups)—mean by evidence and

what they believe ought to be the process for producing

and using evidence, is important.

Existing research has examined how evidence is used

or mobilized and how that might impact on the content

and process of policy-making. Some of these studies

tackle what is meant by evidence and who is considered

a policy-maker (Stewart et al., 2017). Studies rooted in the

Evidence-Based Policy-Making (EBPM) school often focus

on making recommendations about what scholars or

policy-makers could/should do better, rather than exam-

ining meanings and processes of evidence in the real

world (Oliver et al. 2014; Oliver & de Vocht, 2017). Publica-

tions examining policy-makers’perceptions of evidence

are predominantly produced in health policy but less so

in other areas of policy where evidence is often seen as

being more contested (Campbell et al., 2009; Oliver et al.

2014; Vogel et al., 2013; Wallace et al., 2012). There is a

growing body of research on the uses of evidence by

Received: 20 January 2022 Revised: 15 September 2022 Accepted: 24 September 2022

DOI: 10.1111/puar.13567

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribu tion and reproduction in any medium, provided the

original work is properly cited.

© 2022 The Authors. Public Administration Review published by Wiley Periodicals LLC on behalf of American Society for Public Administration.

Public Admin Rev. 2023;83:1037–1050. wileyonlinelibrary.com/journal/puar 1037