Beyond Replication: Secondary Qualitative Data Analysis in Political Science

• Published date: 01 July 2023
• DOI: http://doi.org/10.1177/00104140221139388
• Author: Florian G Kern,Katariina Mustasilta
• Date: 01 July 2023
• Subject Matter: Articles

Article

Comparative Political Studies

2023, Vol. 56(8) 1224–1256

© The Author(s) 2023

Article reuse guidelines:

sagepub.com/journals-permissions

DOI: 10.1177/00104140221139388

journals.sagepub.com/home/cps

Beyond Replication:

Secondary Qualitative

Data Analysis in Political

Science

Florian G Kern

1

and Katariina Mustasilta

2



Abstract

Shared qualitative data –such as interview or focus group transcripts –can be

used for secondary qualitative data analysis (SQDA). Yet, much archived

qualitative data remains unused after primary analysis. Applications and

guidance on how to employ SQDA are rare. We use an example application of

SQDA studying informal institutions and resilience in Sub-Saharan Africa to

show: First, SQDA depends on how primary researchers share ‘raw’qual-

itative data and additional documentation to understand primary context.

Second, deductive and inductive uses of SQDA require varying engagement

with primary data. Third, current practices of participant consent often do not

consider potential SQDA. Fourth, SQDA is not less time-consuming than

primary data research but offers different benef‌its, such as expanding the

comparative sample of cases or avoiding research fatigue of studied com-

munities. Going forward, SQDA requires greater consensus on the instru-

ments (e.g. transcripts and participant consent forms) used by researchers and

further applications of hypothesis-testing and hypothesis-generating designs.

Keywords

secondary qualitative data analysis, qualitative research, transparency,

resilience, Sub-Saharan Africa

1

University of Essex, Colchester, UK

2

Finnish Institute of International Affairs, Helsinki, Finland

Corresponding Author:

Florian G Kern, University of Essex, Wivenhoe Park, Colchester CO4 3SQ, UK.

Email: fkern@essex.ac.uk

Introduction

In recent years, the social sciences have experienced calls for greater

transparency and more replicable and reliable research. In political science,

new principles of Data Access and Research Transparency (DA-RT) have

been incorporated in the ethics guides of the American Political Science

Association (APSA, 2012,2022). Sharing data for replication is an established

convention in quantitative political science (King, 1995). Yet, for qualitative

approaches, the debate on whether and how to practice transparency and data

sharing is on-going (see Büthe et al., 2015;Elman & Kapiszewski, 2014b;

Kapiszewski & Karcher, 2021;Lupia and Elman, 2014), best illustrated by the

Qualitative Transparency Deliberations (QTD) sponsored by the APSA

section for Qualitative and Multi-Method Research (Jacobs et al., 2021).

Beyond increasing transparency and enabling replication, making data

available has additional potential: rather than attempting to replicate an

original f‌inding, the shared primary data (e.g. interview and focus group

transcripts, f‌ield notes, archival texts and audio recordings) can be used by

fellow scholars to answer different research questions through secondary

qualitative data analysis (SQDA) (Büthe & Jacobs, 2015, 56; Elman et al.,

2010,2013;Lupia & Elman, 2014, 22). This is different from the standard

def‌inition of replication, that is, ‘that suff‌icient information exists with which

to understand, evaluate, and a build upon a prior work if a third party could

could replicate the results without any additional information from the author’

(King, 1995, 44). The latter aims to either reproduce a result using the same

data and same question (a standard in observational quantitative social sci-

ence) or to replicate to see if a result holds with the same design but new data

(an increasingly common practice in experimental social science). SQDA,

instead, is about asking new questions with existing, archived qualitative data.

Yet,for a variety of reason s,many scholars remain hesitant about sharing their

qualitative data.

1

Even when archived, the majority of qualitative data remains unused for

SQDA. There is little guidance and applications in political science show-

casing how to prepare qualitative data for secondary analysis and how to run a

re-analysis (with some notable exceptions in more interdisciplinary f‌ields,

especially Hughes & Tarrant, 2019;Irwin et al., 2012;Watkins, 2022). This is

unfortunate, as, in principle, SQDA may have much potential to deepen our

understanding of various social issues by using readily available data. For

instance, assume one researcher has archived interview transcripts from a

study on electoral candidates’intrinsic motivation of running for off‌ice in

Uganda. A second researcher could study candidates’assessment of voter

preferences by basing their investigation (partially or fully) on a secondary

analysis of the shared primary interview transcripts. However, if the only

option of the secondary analyst in the above example is to collect new, primary

Kern and Mustasilta 1225

data, she possibly faces the problem of research fatigue in the study pop-

ulation. This is particularly pertinent when research focuses on vulnerable

populations for which scholars are mindful to not overstretch participants’

time spent providing information. SQDA can also enable researchers with

scarce resources (e.g. in the Global South), who face severe hurdles when

trying to conduct original data collection, to analyse transcripts or f‌ield notes

relevant to their research problem. Moreover, SQDA could provide avenues of

research where original data collection is diff‌icult (as is the case for many

research environments during the COVID-19 pandemic). Ultimately, SQDA

is much akin to what qualitative research has always relied on: the inter-

pretation and analysis of primary sources (say, presidential speeches or ar-

chival texts) –but in SQDA these sources have been recorded or compiled by

another scholar. While SQDA is not a replacement of primary data collection

in political science, both approaches could be used in conjunction.

2

In this paper, we ask: How can political scientists employ archived

qualitative data for secondary data analysis? What information is needed from

the original studies for this purpose beyond the raw qualitative materials? We

show, f‌irst, that SQDA largely depends onhow primary resear chers share their

‘raw’qualitative data, that is, interview and focus group transcripts, as well as

additional documentation to understand primary context; second, that the

deductive and inductive uses of SQDA require a different engagement of the

secondary analyst with the primary data; third, that current ethics practices of

participant consent have to be re-considered to incorporate the consent for

further analysis of archived data; fourth, that SQDA is not less time-

consuming than primary data collection and analysis, but offers different

benef‌its, such as, among other things, expanding the comparative sample of

cases for the analysis or avoiding research fatigue of repeatedly studied

communities.

To illustrate SQDA, and given our own substantive research expertise in

informal institutions (Goist & Kern, 2018;Holzinger et al., 2016,2020;

Mustasilta, 2019,2021), we use an exemplary research project with the re-

search question: how do informal institutions affect community resilience in

Sub-Saharan Africa? We select data from studies which have previously

shared their transcripts from qualitative interviews and focus group discus-

sions on the UK Data Archive (UKDA), overall consisting of 992 interview

transcripts and 151 focus group transcripts.

3

In order to demonstrate chal-

lenges and opportunities when working on a new research problem compared

to the primary study, we have formulated our research question prior to

probing for data availability and then chose studies and data that do not

explicitly focus on our selected research question.

4

This broad focus allows us

to better avoid framing our research question solely on data availability and to

open our sample of data collections up to those from various disciplines, given

that resilience is an interdisciplinary topic of study. In principle, secondary

1226 Comparative Political Studies 56(8)