How qualitative research methods can be leveraged to strengthen mixed methods research in public policy and public administration?

• Published date: 01 May 2023
• Author: Kathryn Hendren,Kathryn Newcomer,Sanjay K. Pandey,Margaret Smith,Nicole Sumner
• Date: 01 May 2023
• DOI: http://doi.org/10.1111/puar.13528

RESEARCH ARTICLE

How qualitative research methods can be leveraged

to strengthen mixed methods research in public policy and

public administration?

Kathryn Hendren

1

| Kathryn Newcomer

2

| Sanjay K. Pandey

2

| Margaret Smith

3

|

Nicole Sumner

2

1

College of Natural Sciences, University of Texas

at Austin, Austin, Texas, USA

2

Trachtenberg School of Public Policy and Public

Administration, George Washington University,

Washington, District of Columbia, USA

3

United State Military Academy, Modern War

Institute at West Point, West Point, New

York, USA

Correspondence

Kathryn Newcomer, Trachtenberg School of

Public Policy and Public Administration, George

Washington University, Washington, DC, USA.

Email: newcomer@gwu.edu

Abstract

Recently, there have been a variety of arguments voiced to encourage that more

attention be given to the role qualitative methods can play in mixed methods

research in public policy and public administration. This article discusses these

claims and describes the benefits of qualitative approaches, and how qualitative

research methods can be leveraged to strengthen mixed methods research in

public administration. We also provide a guide for improving the credibility of

mixed methods research through increasing transparency and discussions of all

methodological decisions. This study is based on a systematic content analysis of

186 mixed methods studies published in public policy and public administration

journals between 2010 and 2018. We found that findings from the quantitative

methods dominated the mixed methods studies, little diversity in data collection

and analysis methods, and frequent failure to integrate insights from both

methods. We also analyzed the 36 qualitative-dominant studies in the sample, and

illuminated seven different ways that authors of qualitative-dominant studies lev-

eraged the qualitative strand to strengthen mixed methods research. We devel-

oped lessons from our analysis of the qualitative-dominant articles on how to

incorporate qualitative methods in a thoughtful manner, articulate a role for each

strand, and effectively support findings with one or more strands.

KEYWORDS

iterative text coding, mixed methods, public administration, qualitative methods, research methods

Evidence for practice

•Mixed methods studies can benefit when quantitative data analyses are inten-

tionally supported and expanded by the use of qualitative methods to obtain

perspectives of study subjects and other stakeholders to flesh out context.

•Effective integration of findings in mixed methods research requires deliberate

dialogue between quantitative and qualitative strands.

•Clear elucidation and application of standards of evidence for qualitative and

quantitative research merit equal attention.

•Transparency in reporting is needed for both qualitative and quantitative

methods in mixed methods studies regarding all aspects of data: sources, collec-

tion, analyses, and reporting.

INTRODUCTION

A major finding about the use of mixed methods in public

policy and public administration research has been the

dominance of quantitative strands over qualitative

strands. Hendren et al. (2018) found that mixed methods

studies across public administration journals overwhelm-

ingly favor quantitative methods, with qualitative strands

Received: 22 October 2020 Revised: 11 May 2022 Accepted: 23 May 2022

DOI: 10.1111/puar.13528

468 © 2022 American Society for Public Administration. Public Admin Rev. 2023;83:468–485.wileyonlinelibrary.com/journal/puar

receiving less emphasis and the qualitative methods less

effectively reported. These findings are in line with

Giddings’s(

2006) caution that the rise and continued pro-

motion of mixed methods research assume that integrat-

ing methods inherently promotes qualitative methods

when in reality, the quantitative methods still dominate.

Such an assumption could lead to a new generation of

researchers who see quantitative-dominated mixed

methods as exemplars of mixed methods scholarship,

and work against the development of and learning about

how to utilize qualitative methods effectively to bolster

findings (Giddings, 2006; Hesse-Biber, 2015; Mason, 2006).

Researchers in a variety of fields have long proposed

that well-developed qualitative strands can add consider-

able value to mixed methods research designs by illumi-

nating the context of and complexities inherent in human

behavior and improving our ability to explain findings

(Creswell et al., 2006; Mason, 2006). Qualitative strands

that emphasize authenticity, plausibility, and credibility

are recommended to add nuance and depth to mixed

methods studies (Brower et al., 2000; Creswell & Plano

Clark, 2017; Greene et al., 1989; Johnson &

Onwuegbuzie, 2004). Strong qualitative strands in mixed

methods research designs have been touted to more fully

capture the benefits of integration, such as encouraging

creativity, gaining new perspectives, clarifying the context

and transferability of findings, and presenting a deeper

and more nuanced understanding (Mason, 2006;

Merriam & Tisdell, 2016; Raimondo & Newcomer, 2017).

Given the value strong qualitative strands can bring to

mixed methods research, our goal is to better understand

and clarify the way qualitative strands can strengthen

mixed methods research in public policy and public

administration. Specifically, we address three questions:

1. How do qualitative-dominant mixed methods studies

differ from quantitative-dominant ones in public pol-

icy and public administration?

2. How can qualitative methods be leveraged to

strengthen mixed methods research in public policy

and public administration?

3. What key reporting elements are needed in a mixed-

method study to increase the strength and credibility

of findings?

In our research, we examine and draw lessons from

mixed methods studies in public policy and public admin-

istration in order to identify promising practices and pro-

vide guidance on how to improve the planning,

execution, and reporting of mixed methods research. We

start with accepting the definition of mixed methods

inquiry as: “the class of research where the researcher

mixes or combines quantitative and qualitative research

techniques, methods, approaches, concepts, or language

into a single study”(Johnson & Onwuegbuzie, 2004,

p. 17). While some observers view the use of two or more

data collection or analysis methods as constituting mixed

methods work, we frame mixed methods research for our

analysis as studies that employ at least one quantitative

and one qualitative research method.

In this article, we first discuss qualitative methods and

arguments given for increasing their use in public policy

and public administration research. Then we describe the

methods we used to analyze mixed methods research in

public policy and public administration. Following the

description of our methods, we present our analyses

which clarify how quantitative- and qualitatively-driven

mixed methods studies have been characterized and how

they differ. Then we discuss how mixed methods research

may be strengthened by a more intentional and strategic

blending of qualitative and quantitative strands. We con-

clude with a set of guidelines that may be used to

improve reporting for mixed methods research.

OPPORTUNITIES FOR ENRICHING MIXED

METHODS RESEARCH WITH QUALITATIVE

METHODS

Before we address how and why qualitative methods can

add value to mixed methods research, the broad range of

qualitative methods available merits delineation. A review

of the literature indicates a variety of approaches for con-

ducting qualitative research. Creswell and Poth (2016)

identify five research approaches to qualitative methods:

narrative, phenomenological, grounded theory, ethno-

graphic, and case study. Merriam and Tisdell (2016) simi-

larly highlight the same five designs and add a sixth:

basic qualitative study. Coffey and Atkinson (1996) find

narrative analysis valuable because the approach “facili-

tates the exploration of content in interviews [and] field

notes”(p. 80). Denzin and Lincoln (2011) further explain

how qualitative research uses a variety of empirical mate-

rials: case study, personal experience, introspection, life

story, interview, artifacts, cultural texts, observation, and

historical information. The variety of qualitative

approaches and data sources enhance the flexibility with

which qualitative methods can be applied.

Although qualitative analysis includes a variety of

approaches, researchers generally agree on the founda-

tional value of careful and transparent coding which

“enables the researcher to recognize and contextualize

qualitative data”(Coffey & Atkinson, 1996, p. 45). Creswell

and Poth (2016) designate three baseline analytic steps:

“coding the data (reducing the data into meaningful seg-

ments and assigning names for the segments), combining

the codes into broader categories or themes, and dis-

playing and making comparisons in the data graphs,

tables, and charts”(p. 183–184). They further emphasize a

dynamic interaction between the researcher and the data

in which the researcher repeatedly engages with and

assesses the data in a fluid rather than rigid manner

(Creswell & Poth, 2016, p. 185). Coffey and Atkinson

(1996) also explain that multiple coding tools contribute

PUBLIC ADMINISTRATION REVIEW 469